rps: Inspect GRE encapsulated packets to get flow hash

Crack open GRE packets in __skb_get_rxhash to compute 4-tuple hash on
in encapsulated packet.  Note that this is used only when the
__skb_get_rxhash is taken, in particular only when the device does
not compute provide the rxhash (ie. feature is disabled).

This was tested by creating a single GRE tunnel between two 16 core
AMD machines.  200 netperf TCP_RR streams were ran with 1 byte
request and response size.

Without patch: 157497 tps, 50/90/99% latencies 1250/1292/1364 usecs
With patch: 325896 tps, 50/90/99% latencies 603/848/1169

Signed-off-by: Tom Herbert <therbert@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

diff --git a/net/core/dev.c b/net/core/dev.c
index 4bee9a9aeef639bcad82923f5e08806935595e76..a8d91a5dd9094afaf4346bbf8dee55aad7b57447 100644 (file)
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2570,6 +2570,28 @@ again:
        }
 
        switch (ip_proto) {
+       case IPPROTO_GRE:
+               if (pskb_may_pull(skb, nhoff + 16)) {
+                       u8 *h = skb->data + nhoff;
+                       __be16 flags = *(__be16 *)h;
+
+                       /*
+                        * Only look inside GRE if version zero and no
+                        * routing
+                        */
+                       if (!(flags & (GRE_VERSION|GRE_ROUTING))) {
+                               proto = *(__be16 *)(h + 2);
+                               nhoff += 4;
+                               if (flags & GRE_CSUM)
+                                       nhoff += 4;
+                               if (flags & GRE_KEY)
+                                       nhoff += 4;
+                               if (flags & GRE_SEQ)
+                                       nhoff += 4;
+                               goto again;
+                       }
+               }
+               break;
        default:
                break;
        }