x86, rwsem: Stay on fast path when count > 0 in __up_write()

When count > 0 there is no need to take the call_rwsem_wake path.  If
we did take that path, it would just return without doing anything due
to the active count not being zero.

Signed-off-by: Michel Lespinasse <walken@google.com>
LKML-Reference: <201007202219.o6KMJj9x021042@imap1.linux-foundation.org>
Acked-by: David Howells <dhowells@redhat.com>
Cc: Mike Waychison <mikew@google.com>
Cc: Suleiman Souhlal <suleiman@google.com>
Cc: Ying Han <yinghan@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>

diff --git a/arch/x86/include/asm/rwsem.h b/arch/x86/include/asm/rwsem.h
index 606ede126972e568b992268da3df6f64b7e98480..5bf5e04e497fa83bf81f58d765c8e40d4d9f67ea 100644 (file)
--- a/arch/x86/include/asm/rwsem.h
+++ b/arch/x86/include/asm/rwsem.h
@@ -216,9 +216,8 @@ static inline void __up_write(struct rw_semaphore *sem)
        rwsem_count_t tmp;
        asm volatile("# beginning __up_write\n\t"
                     LOCK_PREFIX "  xadd      %1,(%2)\n\t"
-                    /* tries to transition
-                       0xffff0001 -> 0x00000000 */
-                    "  jz       1f\n"
+                    /* subtracts 0xffff0001, returns the old value */
+                    "  jns        1f\n\t"
                     "  call call_rwsem_wake\n"
                     "1:\n\t"
                     "# ending __up_write\n"