From: Thibaut VARÈNE Date: Thu, 21 Jun 2018 09:04:37 +0000 (+0200) Subject: phase1: do not exceed nproc build concurrency X-Git-Tag: v1~150 X-Git-Url: http://git.lede-project.org./?a=commitdiff_plain;h=2ad04784a7ea7ab317ac7e0e670996b920fd72ec;p=buildbot.git phase1: do not exceed nproc build concurrency Contrary to popular belief, exceeding the number of available CPU cores in parallel builds (by running e.g. 'make -j$(($nproc+1))' where 'nproc' is the number of active CPUs on the system) brings no performance benefit, and may in fact negatively affect build times. This performance hit can be further aggravated by the extra memory pressure resulting from the extraneous jobs. See: - https://blog.regehr.org/archives/1416 - http://blog.stuffedcow.net/2011/08/hyperthreading-performance/ This is particularly relevant in two distinct cases: - on hyper-threaded systems, where half of the active CPUs are separate threads on the same physical core; - on virtualized guests systems, where the host physical CPUs are already affected by system overhead not visible to the guest. Signed-off-by: Thibaut VARÈNE --- diff --git a/phase1/master.cfg b/phase1/master.cfg index e00a896..cb0e709 100644 --- a/phase1/master.cfg +++ b/phase1/master.cfg @@ -272,7 +272,7 @@ def GetVersionPrefix(props): def GetNumJobs(props): if props.hasProperty("slavename") and props.hasProperty("nproc"): - return ((int(props["nproc"]) / (max_builds[props["slavename"]] + other_builds)) + 1) + return (int(props["nproc"]) / (max_builds[props["slavename"]] + other_builds)) else: return 1