sched: fix broken SMT/MC optimizations

On a four package system with HT - HT load balancing optimizations were
broken.  For example, if two tasks end up running on two logical threads
of one of the packages, scheduler is not able to pull one of the tasks
to a completely idle package.

In this scenario, for nice-0 tasks, imbalance calculated by scheduler
will be 512 and find_busiest_queue() will return 0 (as each cpu's load
is 1024 > imbalance and has only one task running).

Similarly MC scheduler optimizations also get fixed with this patch.

[ mingo@elte.hu: restored fair balancing by increasing the fuzz and
                 adding it back to the power decision, without the /2
                 factor. ]

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 1845b2e99a871eca1c2bf439468102d4f5f2fecc..ba78807eab919e5c9229c09476251df643d6012e 100644 (file)
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -681,7 +681,7 @@ enum cpu_idle_type {
 #define SCHED_LOAD_SHIFT       10
 #define SCHED_LOAD_SCALE       (1L << SCHED_LOAD_SHIFT)
 
-#define SCHED_LOAD_SCALE_FUZZ  (SCHED_LOAD_SCALE >> 1)
+#define SCHED_LOAD_SCALE_FUZZ  SCHED_LOAD_SCALE
 
 #ifdef CONFIG_SMP
 #define SD_LOAD_BALANCE                1       /* Do load balancing on this domain. */

diff --git a/kernel/sched.c b/kernel/sched.c
index 5fecbbba12acff129654b7f0ab67c2cadabe6b4e..d96030db8ff7b21e4cc592b7dafeef1af8a98cd4 100644 (file)
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -2517,7 +2517,7 @@ group_next:
         * a think about bumping its value to force at least one task to be
         * moved
         */
-       if (*imbalance + SCHED_LOAD_SCALE_FUZZ < busiest_load_per_task/2) {
+       if (*imbalance + SCHED_LOAD_SCALE_FUZZ < busiest_load_per_task) {
                unsigned long tmp, pwr_now, pwr_move;
                unsigned int imbn;