Diverting trains of thought, wasting precious time Fri, 05 Sep 2025 Interposing on clone() system calls in-process, from Linux userspace (https://www.humprog.org/~stephen/blog/2025/09/05/) I've written before about a simple user-space approach to intercepting and modifying system calls (https://www.humprog.org/%7Estephen/blog/2021/10/14#syscall-tracing-in-process), by turning system calls into SIGILL and then executing the perhaps-modified system call in the signal handler. I also mentioned that handling clone(), Linux's system call for creating a new thread, is challenging in this context. Let's imagine we want our handler firstly to print out a message, then to do the original clone(), and then return to the original caller (twice, naturally!). Since the clone() in the signal handler context will completely replace the stack, in the child thread the system call context, i.e.the signal frame saved on the original stack, is gone! How do we make that cloned thread return to the caller, i.e.to the place where the program wants the thread actually to begin its execution? I mentioned I would save the detail for another post. This is that post. clone() is a confusing system call because its C library wrapper differs significantly from the actual system call. Whereas the C library wrapper has seven arguments, including a “run this code” function pointer argument fn... int clone(int (fn)(void ), void stack, int flags, void arg, pid_t parent_tid, void tls, pid_t *child_tid); ... the actual system call has only five arguments and the code pointer is conspicuously missing. long clone(unsigned long flags, void *stack, int parent_tid, int child_tid, unsigned long tls); The reason is that the clone() system call proper is like fork(): in both parent and child, execution returns to the calling code, and each of them can tell, from the return value, which one they are. The C library wraps the system call in some hand-crafted assembly such that in the child, control jumps to the function whose addressed was passed in. Let's consider a vanilla use of clone() that accepts a new stack. (Technically it's optional to provide a new stack, but doing so is normal and is what makes life tricky for us. So we will ignore the cases where this isn't done.) We have two stacks. calling stack new stack .___________. | | | ... | caller state | | |___________|<--sp at clone .___________.<--child sp as given to clone : : : : : ... : unused stack space : ... : unused stack space For simplicity I will assume, as is usually the case, that the new thread's execution begins with nothing on the stack, i.e.the caller hasn't pre-populated the stack in any way. Now what do we have in our signal-handling context where we want to do the clone? If we just do the clone like we would do any other system call, it would look like this. Let's suppose that we have some function called do_syscall() that is now responsible for doing the clone. calling stack new stack | ... | caller state |___________|<--sp at trap | | | | signal frame |___________| | | frame for do_syscall | | |___________|<--sp at clone .___________.<--child sp as given to clone : : : : : ... : : ... : The problem is that now, when we complete the clone(), although we've created a new thread, our program counter is still in do_syscall(). We're a long way from the code that the original caller, i.e.the code at the trap site, wanted to run using this new stack. And in the child, our parent thread's stack is no longer available! The parent may happen to have been scheduled first and proceeded to tear down its stack, before the child ran at all. We have to get our child thread to the intended (calling) code, without reference to the parent stack. We could hand-craft do_syscall() in assembly, like how the libc clone() wrapper code is written. A single coherent blob of assembly language, performing the entire do_syscall() logic, could do this, carefully transferring control back to the caller without using the stack after the clone. However, that would be a big blob of assembly code. The entire signal-handling logic, which is our return path, needs to be in assembly too. I would much rather use a C compiler. If we use a C compiler, the compiler-generated code will assume it has a stack to work with! There's no way to tell it not to. Also, it does not know that the stack gets nuked across the call, and there is no way to enlighten the compiler about this, even with extended inline assembly. So, things can easily go very wrong. For example, in both parent and child, do_syscall() will pretty soon want to return through the signal handler by grabbing a return address off the stack. That's fine for the parent thread, but the child's stack does not contain this information. Could we make the child just jump immediately to the trap site's successor instruction? We could easily keep that instruction address in a register across the system call, to be sure it remains available when the stack is nuked. I considered this, but it has an annoying lack of uniformity. For example, in my do_syscall() (actually a bunch of functions, not just one) I want to be able to add pre- and post-actions to any system call. (For a successful clone() that means one pre-action but two post-actions.) That creates an incentive to return through a common code path. If we just jump away, we would have to special-case the post-handling for clone() in the child. Also, there is at least one other call, clone3(), that needs similar special-casing. Instead, I found a way to do it that keeps the amount of assembly code minimal and keeps a mostly common return path. It's not rocket science, just some careful thought. What we do is pre-populate the new stack with sufficient content that it can successfully return through the usual signal handling return path. It means that our single SIGILL, triggered by a trapped clone() syscall, will go on to generate two sigreturn() syscalls, because we return from the signal handler in both the parent and the child thread. calling stack new stack | ... | caller state |___________|<--sp at trap .___________.<--caller's intended child sp | | | _ | | | signal frame | | copy of signal frame |___________| |___________| | | frame for do_syscall | _ | copy of frame for do_syscall | | | | |___________|<--sp at clone |___________|<--child sp really given to clone : : : : : ... : : ... : This pre-populated content is more-or-less a copy of the existing signal and stack frames. Spoiler: as you might guess, the copy is not an exact copy. Here's what the x86-64 assembly snippet for doing the system call looks like. Initially r12 contains gsp, a pointer to a structure that describes a generic system call, including copies of all its argument values. This is passed, together with a few other arguments, to an out-of-line helper, copy_to_new_stack(), which is responsible for doing the pre-populating. It is also written in C and uses the normal System V ABI. The new stack pointer we give to the clone() call is, correspondingly, fixed up to point a little way down the caller's memory region. We must do a similar fix up to the base pointer (rbp) which requires one extra gyration. pushq %rbp # Preemptively save %rbp (see below). movq %rsp, %rcx # rcx will form arg3 (sp_at_clone) of our helper SysV call (copy_to_new_stack) # Now we've locked in our stack pointer position as it will be for the syscall. Anything we push from here needs a corresponding pop before the syscall. # PUSH_SYSV_CLOBBERS_EXCEPT_RCX # pushes rax, rdx, rsi, rdi, r8, r9, r10, r11 movq %r12, %r8 # put gsp in r8, i.e. arg4 of SysV call to copy_to_new_stack callq copy_to_new_stack # copy_to_new_stack(flags_unused, new_stack, # parent_tid_unused, rsp_on_clone, gsp) # ... returns actual new stack pointer to give to clone POP_SYSV_CLOBBERS_EXCEPT_RCX_AND_RAX # pops r11, r10, r9, r8, rdi, rsi, rdx movq %rax, %rsi # copy_to_new_stack returned the actual new stack to use popq %rax addq %rsi, %rbp # Make bp point into the same offset in the child stack... # ... as it currently points to in ours. Then preemptively... subq %rsp, %rbp # ... set this as the BP. In the non-child case we restore the # parent BP that we pushed above. This is easier than updating # the bp only in the child, after the clone, because then we no # longer have the parent stack as a reference. syscall We've potentially cloned... Immediately test: did our stack actually get replaced? cmpq %rsp, %rsi je 1f # If taken, it means we are the child with a new stack. parent case popq %rbp # Else we are the parent, so restore the correct (parent) bp jmp 2f 1:# child case addq $0x8, %rsp # Discard the unwanted bp saved slot on the child stack (uninitialized!) jmp 2f # For all our locals, either - the compiler chose a register not clobbered by the syscall (declared below), or - the compiler put them on the stack and we copied/rewrote them appropriately. So things should continue to Just Work from here... in particular we should have a working %rbp and %rsp in both parent and child. # 2:# parent and child both continue here nop The Linux kernel does not mind at all that a single signal generates two sigreturn() calls. I suppose that might not be portable or durable, but I'd be surprised if it ever stopped working. It is also pleasingly analogous to how the kernel first enters user mode on an Intel CPU: it does an iret, having constructed an exception frame on the stack, despite there never having been an interrupt creating that frame. There are three added hairinesses. Firstly, the kernel is very fussy about the alignment of the fake signal frame that we create on the new stack. In particular the floating-point state area within the signal frame may have stricter alignment requirements than the overall signal frame structure appears to. On 32- and 64-bit Intel, 64 bytes seems to be sufficient, although I forget precisely which cryptic comment in kernel code led me to that number. This affects us as follows: when allocating space on the new stack, we must ensure it begins at the same alignment-modulo-64 that the original signal frame did. (If the kernel does not like the signal frame we create, it will print a “bad frame in rt_sigreturn” message, visible via dmesg. This was invaluable for me to debug this.) Secondly, doing the equivalent of the above assembly on 32-bit Intel is trickier owing to the shortage of registers. I have had GCC complain spuriously that it could not solve the register constraints, particularly once the assembly snippet starts doing things to ebp. This seemed sensitive to what one might think are innocuous changes, such as turning on -fno-strict-aliasing, using asm to get the value of %ebp, introducing memcpy() calls, and so on. I temporarily resorted to sneaky workarounds, such as refraining from declaring certain assembly outputs to the compiler. Happily, after rewriting the code with various simplifications, this GCC complaint seems to have gone away, although the spectre of its re-emergence is not entirely satisfactory. Finally, there's the big thing I've been neglecting: how do we do the copy? It's not quite a straightforward memcpy(). When we construct the new stack, any pointers that are logically pointing “within” the stack need to be fixed up, to point within the new stack rather than across to the old one (which may no longer exist, when these pointers are later used!). At a minimum, this should include the frame pointer saved on entry to do_syscall(), since we need to restore it along the return path. It should also include the sp of the signal frame itself, since it is only by the fixed-up value of this that we communicate the new stack pointer to the caller at the trap site. The sigreturn() call, rather than the clone() call, is now the path by which this register gains its new value in the calling context. How can we know exactly which words within the copied stack are pointers, and therefore need fixing up (if they point within the old stack)? In general we can't be sure we know about all such pointers: the compiler is free to use a stack slot in do_syscall() to save a pointer that is needed along the return path. In the child copy, such a pointer needs to point to the child stack. (Remember: we haven't told the compiler that we're nuking the stack, and we're going behind its back by copying our current stack frame to a new location.) The compiler needn't be friendly or straightforward; in principle it could encode that pointer in any way whatsoever, such as XORed with 42. So this is intractable in general, without more information from the copmiler. For now, I have coded up a giant hack, rather like a conservative garbage collector: copy_to_new_stack() is a wordwise copy loop which rewrites any word that looks like a pointer within the parent stack range. That is overapproximate, of course—it runs a risk of corrupting some innocent integer that happens to look like such a pointer. We could instead be underapproximate, fixing up only the ABI-prescribed on-stack pointers and avoid modifying anything else, like the hypothetically possible compiler temporaries or, in general, any compiler-managed intra-stack pointers. It might mean that we don't work under a very wacky compiler that saves temporary pointers unnecessarily, but would mean that we don't corrupt innocent user data. That is probably the sensible trade-off here. Of course the maximalist option is to have our assembly snippet take care of returning from do_syscall() itself. That is what we ruled out at the start of this post, on grounds of too much assembly language. In a more realistic implementation, do_syscall() is not a single function. In mine it is really handle_sigill(), do_generic_syscall_and_fixup(), is_special_syscall(), fixup_sigframe_for_return() and __systrap_post_handling(). Our assembly code would have to return through a nest of these functions, all the way back to the sigreturn, to be sure not to use any of these compiler-managed pointers. (It would not suffice just to return from the innermost out-of-line call.) Since our initial premise was that we mostly wanted to write our handler logic in C, this option is, by definition, more assembly than we are willing to write. Ideally we would be able to inform the compiler about stack-zapping code like this. But also, ideally the compiler would inform us: about exactly how it is using the stack and where it might be stashing any pointers. The absence of such information is essentially what prevents, at present, a garbage collector like Boehm-Demers-Weiser (https://www.hboehm.info/gc/) from being precise (https://mmref.readthedocs.io/en/latest/glossary/e.html#term-exact-garbage-collection) rather than conservative. Compilers mediate almost all storage to the stack and registers, and they “know” what they are storing in these places—they just don't tell us. They should! I'm working on it.... The code for all this is in my libsystrap (https://github.com/stephenrkell/libsystrap) repository—at the time of writing, most of what I've written about is in src/do-syscall.h. [/devel (https://www.humprog.org/%7Estephen/blog/devel)] [all entries (https://www.humprog.org/%7Estephen/blog-all.html)] permalink (https://www.humprog.org/%7Estephen/blog/2025/09/05#clone-calling)contact (https://www.humprog.org/~stephen/#contact) • http://blosxom.sourceforge.net/ validate this page (http://validator.w3.org/check?uri=https://www.humprog.org/%7Estephen/blog)