ksm: cleanup stable_node chain collapse case

Patch series "KSMscale cleanup/optimizations".

There are no fixes here it's just minor cleanups and optimizations.

1/3 removes makes the "fix" for the stale stable_node fall in the
    standard case without introducing new cases.  Setting stable_node to
    NULL was marginally safer, but stale pointer is still wiped from the
    caller, this looks cleaner.

2/3 should fix the false positive from Dan's static checker.

3/3 is a microoptimization to apply the the refile of future merge
    candidate dups at the head of the chain in all cases and to skip it in
    one case where we did it and but it was a noop (to avoid checking if
    it was already at the head but now we've to check it anyway so it got
    optimized away).

This patch (of 3):

When the stable_node chain is collapsed we can as well set the caller
stable_node to match the returned stable_node_dup in chain_prune().

This way the collapse case becomes indistinguishable from the regular
stable_node case and we can remove two branches from the KSM page
migration handling slow paths.

While it was all correct this looks cleaner (and faster) as the caller has
to deal with fewer special cases.

Link: http://lkml.kernel.org/r/20170518173721.22316-2-aarcange@redhat.com
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Cc: Evgheni Dereveanchin <ederevea@redhat.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Petr Holasek <pholasek@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Gavin Guo <gavin.guo@canonical.com>
Cc: Jay Vosburgh <jay.vosburgh@canonical.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/ksm.c b/mm/ksm.c
index 5621840401c05ef439764bdf74f59fed8b5fbc3f..64b054641f391ae3dca7bfdf7a6de0adba673a89 100644 (file)
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -1392,14 +1392,18 @@ static struct stable_node *stable_node_dup(struct stable_node **_stable_node,
                        ksm_stable_node_chains--;
                        ksm_stable_node_dups--;
                        /*
-                        * NOTE: the caller depends on the
-                        * *_stable_node to become NULL if the chain
-                        * was collapsed. Enforce that if anything
-                        * uses a stale (freed) stable_node chain a
-                        * visible crash will materialize (instead of
-                        * an use after free).
+                        * NOTE: the caller depends on the stable_node
+                        * to be equal to stable_node_dup if the chain
+                        * was collapsed.
                         */
-                       *_stable_node = stable_node = NULL;
+                       *_stable_node = found;
+                       /*
+                        * Just for robustneess as stable_node is
+                        * otherwise left as a stable pointer, the
+                        * compiler shall optimize it away at build
+                        * time.
+                        */
+                       stable_node = NULL;
                } else if (__is_page_sharing_candidate(found, 1)) {
                        /*
                         * Refile our candidate at the head
@@ -1505,7 +1509,11 @@ again:
                 * not NULL. stable_node_dup may have been inserted in
                 * the rbtree instead as a regular stable_node (in
                 * order to collapse the stable_node chain if a single
-                * stable_node dup was found in it).
+                * stable_node dup was found in it). In such case the
+                * stable_node is overwritten by the calleee to point
+                * to the stable_node_dup that was collapsed in the
+                * stable rbtree and stable_node will be equal to
+                * stable_node_dup like if the chain never existed.
                 */
                if (!stable_node_dup) {
                        /*
@@ -1623,15 +1631,13 @@ out:
 replace:
        /*
         * If stable_node was a chain and chain_prune collapsed it,
-        * stable_node will be NULL here. In that case the
-        * stable_node_dup is the regular stable_node that has
-        * replaced the chain. If stable_node is not NULL and equal to
-        * stable_node_dup there was no chain and stable_node_dup is
-        * the regular stable_node in the stable rbtree. Otherwise
-        * stable_node is the chain and stable_node_dup is the dup to
-        * replace.
+        * stable_node has been updated to be the new regular
+        * stable_node. A collapse of the chain is indistinguishable
+        * from the case there was no chain in the stable
+        * rbtree. Otherwise stable_node is the chain and
+        * stable_node_dup is the dup to replace.
         */
-       if (!stable_node || stable_node_dup == stable_node) {
+       if (stable_node_dup == stable_node) {
                VM_BUG_ON(is_stable_node_chain(stable_node_dup));
                VM_BUG_ON(is_stable_node_dup(stable_node_dup));
                /* there is no chain */
@@ -1676,13 +1682,13 @@ chain_append:
                stable_node_dup = stable_node_any;
        /*
         * If stable_node was a chain and chain_prune collapsed it,
-        * stable_node will be NULL here. In that case the
-        * stable_node_dup is the regular stable_node that has
-        * replaced the chain. If stable_node is not NULL and equal to
-        * stable_node_dup there was no chain and stable_node_dup is
-        * the regular stable_node in the stable rbtree.
+        * stable_node has been updated to be the new regular
+        * stable_node. A collapse of the chain is indistinguishable
+        * from the case there was no chain in the stable
+        * rbtree. Otherwise stable_node is the chain and
+        * stable_node_dup is the dup to replace.
         */
-       if (!stable_node || stable_node_dup == stable_node) {
+       if (stable_node_dup == stable_node) {
                VM_BUG_ON(is_stable_node_chain(stable_node_dup));
                VM_BUG_ON(is_stable_node_dup(stable_node_dup));
                /* chain is missing so create it */