elf: init pt_regs pointer later
authorAlexey Dobriyan <adobriyan@gmail.com>
Tue, 14 May 2019 22:43:54 +0000 (15:43 -0700)
committerLinus Torvalds <torvalds@linux-foundation.org>
Wed, 15 May 2019 02:52:50 +0000 (19:52 -0700)
commit249b08e4e504d4c54eda3453c9c97edbafa51401
treeec52500abd913e84b8f5072975e09dc472dd244f
parentd8e7cb39acc66316106ef8bda9b76f9f3cbbcad0
elf: init pt_regs pointer later

Get "current_pt_regs" pointer right before usage.

Space savings on x86_64:

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-180 (-180)
Function                           old     new   delta
load_elf_binary                   5806    5626    -180 !!!

Looks like the compiler doesn't know that "current_pt_regs" is stable
pointer (because it doesn't know ->stack isn't) even though it knows
that "current" is stable pointer.  So it saves it in the very beginning
and then tries to carry it through a lot of code.

Here is what happens here:

load_elf_binary()
...
mov rax,QWORD PTR gs:0x14c00
mov r13,QWORD PTR [rax+0x18] r13 = current->stack
call kmem_cache_alloc # first kmalloc

[980 bytes later!]

# let's spill that sucker because we need a register
# for "load_bias" calculations at
#
# if (interpreter) {
# load_bias = ELF_ET_DYN_BASE;
# if (current->flags & PF_RANDOMIZE)
# load_bias += arch_mmap_rnd();
# elf_flags |= elf_fixed;
# }
mov QWORD PTR [rsp+0x68],r13

If this is not _the_ root cause it is still eeeeh.

After the patch things become much simpler:

mov rax, QWORD PTR gs:0x14c00 # current
mov rdx, QWORD PTR [rax+0x18] # current->stack
movq [rdx+0x3fb8], 0 # fill pt_regs
...
call finalize_exec

Link: http://lkml.kernel.org/r/20190419200343.GA19788@avx2
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Tested-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
fs/binfmt_elf.c