mm: vmscan: do not OOM if aborting reclaim to start compaction

During direct reclaim it is possible that reclaim will be aborted so that
compaction can be attempted to satisfy a high-order allocation.  If this
decision is made before any pages are reclaimed, it is possible that 0 is
returned to the page allocator potentially triggering an OOM.  This has
not been observed but it is a possibility so this patch addresses it.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: Dave Jones <davej@redhat.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Andy Isaacson <adi@hexapodia.org>
Cc: Nai Xia <nai.xia@gmail.com>
Cc: Johannes Weiner <jweiner@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 866ab27c52a5d0ac809efe48c15205fd45e37ec4..cb68c53db4ecc19359a77192632e149db890265f 100644 (file)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2263,6 +2263,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
        struct zoneref *z;
        struct zone *zone;
        unsigned long writeback_threshold;
+       bool should_abort_reclaim;
 
        get_mems_allowed();
        delayacct_freepages_start();
@@ -2274,7 +2275,8 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
                sc->nr_scanned = 0;
                if (!priority)
                        disable_swap_token(sc->target_mem_cgroup);
-               if (shrink_zones(priority, zonelist, sc))
+               should_abort_reclaim = shrink_zones(priority, zonelist, sc);
+               if (should_abort_reclaim)
                        break;
 
                /*
@@ -2342,6 +2344,10 @@ out:
        if (oom_killer_disabled)
                return 0;
 
+       /* Aborting reclaim to try compaction? don't OOM, then */
+       if (should_abort_reclaim)
+               return 1;
+
        /* top priority shrink_zones still had more to do? don't OOM, then */
        if (global_reclaim(sc) && !all_unreclaimable(zonelist, sc))
                return 1;