Diverting trains of thought, wasting precious time Fri, 02 May 2025 Writing a preloadable malloc in Rust, using MMTk (https://www.humprog.org/~stephen/blog/2025/05/02/) Obligatory banner-esque note: between now and August-ish I'm prospecting for crowdfunding that might support my research and writing in the future, as I move to being only a part-time academic. Read more [in my earlier post (https://www.humprog.org/~stephen/blog/2025/02/28#the-mess-im-in), or you can directly drop me a no-commitment HTML form submission (https://www.humprog.org/cgi-bin/support-stephen-in-principle/) if you think you might be able to contribute, or e-mail me (mailto:stephen@humprog.org). Thanks so much to everyone who has done so already! Also, moral support is support too, and is appreciated. :-)] I recently(-ish) spent a while on sabbatical at the ANU's School of Computing (https://comp.anu.edu.au/) in Canberra with the excellent people there, to work on some proof-of-concept integrations of liballocs (https://github.com/stephenrkell/liballocs/) with MMTk (https://mmtk.io/), the memory management toolkit. I enjoyed all my interactions with the many excellent people there, and am especially grateful to my hosts Tony Hosking and Steve Blackburn. In contrast to its JikesRVM origins (https://www.jikesrvm.org/), the current incarnation of MMTk is written in Rust. That makes it feasible to integrate it in a wide variety of language runtimes: quite a departure from the RVM concept, which was relatively standalone. Indeed, many such integrations have already been done! Soon (or already?) both Julia and Ruby will be optionally using it in their mainline releases, There are also ports of the most challenging runtimes out there, including OpenJDK, V8 and ART. Needless to say, there is a lot of sophistication in MMTk and and a huge investment of development effort. If you're not familiar, I'd definitely recommend taking a look. All this aligns well with my interests with liballocs. Although so far that has focused on C compilers (mostly) and adjacent toolchains (a little), I want to bring all languages under the umbrella. I want to show, generally, that language VMs can be retrofitted to work with it, and specifically that the liballocs meta-level interface can be implemented over their heaps. This needs to be doable without major performance sacrifice and without far-reaching code changes. That's already been proof-of-concepted for malloc-style heaps, but not for any garbage-collected heaps. I need to start small. In this post I will quickly revert to type and focus on a really simple target: a malloc-style memory management interface, albeit implemented using MMTk. Can we plumb this into a liballocs-enabled process? If so, that's a useful first step. It's a modest goal. Happily the answer is yes. But in yet more reversion to type, most of the difficulties are of plumbing: linking, Rust and linking Rust. So in this post we won't get as far as clever memory management techniques. In fact my free() is currently a no-op. How do we create a malloc in Rust and package it as a shared library? That way, in dynamically linked ELF environments, we can conveniently preempt the C library's malloc using their own, via LD_PRELOAD. Replacing malloc reliably is a tricky business no matter what language we are working in, essentially because of reentrancy. malloc is entwined in the lower levels of virtually every process's runtime, through standard libraries—whether C, Rust or most other languages'—and also through the dynamic linker. The latter matters to us: although no bindings within our Rust code are linked dynamically, the wider process is binding to it dynamically, because it's in a shared library. I'm new to Rust and this was a learning exercise for me. Spoiler alert: I've found ways of solving all the issues I encountered. Some of the solutions are not pretty; I've filed one bug (https://sourceware.org/bugzilla/show_bug.cgi?id=32443) against GNU binutils and argued for one closed WONTFIX (https://sourceware.org/bugzilla/show_bug.cgi?id=16693) to be reconsidered (I have a workaround but it's not nice). I have some things to report in Rust too (sanity check pending). Such difficulties are par for the course, for a niche task such as this. Avoiding direct reentrant use of global malloc within Rust libraries Step one is to ensure our new global malloc does not depend circularly on the global malloc. Since various Rust standard libraries, which MMTk uses, are implemented to consume a malloc, we need a way to divert those to a private malloc, rather than the global one we're implementing. This is pretty easy in Rust. // recipe stolen from rustc's library/std/src/alloc.rs use jemallocator::Jemalloc; #[global_allocator] static GLOBAL: Jemalloc = Jemalloc; If using Cargo it's necessary to add jemalloc to the [dependencies] block of Cargo.toml. Avoiding transitive reentrant use of global malloc The above sorts out the “immediate” cases, but there's also some “transitive” cases: Rust libraries that consume some other API whose implementation—C code, most likely—itself calls the global malloc. Being outside Rust, these calls aren't diverted by the global_malloc step we just took. One problem is that no well-defined contract exists saying which library functions, in C or in Rust, might call malloc. An implementor of malloc really wants to know this, to avoid using them or else to ensure they are bound to a malloc other than the one being implemented. The particular indirect case that bit me is a call from Rust libraries to the C library's __cxa_thread_atexit_impl, which can then call the global malloc. There could be other such C library calls too. This is really a linking problem, of the kind tools like Knit (https://www.flux.utah.edu/paper/reid-osdi00) would be good at solving, in a better world. As things are, using a vanilla linker, we lack fine-grained control of the linkage graph. We do have a little influence on the binding of outgoing references to malloc: we can play with link order, symbol visibilities, and binding modes like -Bsymbolic. But these mechanisms all turn out to be too coarse-grained because, at least when linking a single binary, they don't let us talk about individual definitions or references to a given symbol; we only get to mention the symbol itself, by name, and treat all uses of it the same. (Our scenario is Figure 1(c) in the Knit paper. I'll come on to how to link Rust code with -Bsymbolic below, although it turns out we don't actually need that in this case.) Even if we had Knit, we would have a delineation problem. We want to say something like “downstream of our malloc, don't bind to the global malloc” (because that would be us! and hence circular). But how do we define “downstream”? DSO granularity is too coarse: then the whole of libc is downstream, which is more than we want. Ideally most of libc should use our malloc. The exception should be just the small subset of libc that we ourselves need to use. Knit could hypothetically instead give us .o granularity, e.g.imagining something like thread_atexit.o within libc, its bindings could be adjusted distinctly from others'. That could be a plausible way to approach our reentrancy problem. But it's a bit of a semantic minefield: rather like monkey-patching (monkey-linking?), it's sensitive to the change-prone internals of a codebase (libc) that we'd much rather treat as a black box. And it isn't expressible in our scenario anyway, of course: inside DSOs like libc, we lack this Knit-style view and indeed those fine-grained .o boundaries have been erased. Another way to approach the problem would be to create our malloc.so to be entirely self-contained. That would mean it does not not make any demands of the surrounding libc interfaces. This is quite maximalist: we are applying a “don't use the global symbol” rule, within our malloc's implementation, for all symbols downstream, not just for malloc. Whereas our “all of libc” scenario above. was about rewiring all of libc to use a secondary malloc (and that was more rewiring of libc than we ideally wanted), this is about rewiring all of our malloc, including its transitive dependencies, into a secondary libc. That is more rewiring of our malloc than we ideally want! But both of these “extreme rewirings” do avoid reentrant calls to our malloc. Folklore does have an approach to building self-contained shared libraries, by linking statically a second copy of the C library (or other system libraries). In so doing, we meet all libc needs of our malloc-defining code in one fell swoop. It's not hard to do this, although again, in the absence of Knit-like tools, one wart is that we need to call our malloc definition something else, like mymalloc, at least until we've done the static link step. After that, we can rename it. So it's not unusual to see the first step as a relocatable link: something like this. $ ld -r -o mymalloc.os mymalloc.o -Bstatic libc.a $ objcopy --redefine-sym malloc=libc_malloc --redefine-sym mymalloc=malloc mymalloc.os (Here .os is something I pronounce “ose”, like “got any 'ose?” (https://www.youtube.com/watch?v=gi_6SaqVQSw) but that might not be canonical.) If you try the above you'll find two complications: you need a position-independent libc.a, and depending on the libc, you may need a version script too. (Glibc makes this difficult. A PIC version is built as libc_pic.a, but distributions tend not to include this. It is in the build tree if you do the build yourself, though. To get a version script, the best way I found was to piece it together from its version files in the build tree. Ask me how! I hacked together some awk scripts to do this but there may be a nicer way.) Once you've tackled those, you can make self-contained shared objects that statically link glibc code. If we did this in our case, then our Rust code would pull in __cxa_thread_atexit_impl statically, and bind to our static copy of malloc and everything else. We should also give all these extra pulled-in definitions hidden visibility (see below), so that they do not interfere with the bindings of other dynamic objects. Once we've done that, we're free to rename mymalloc to malloc and we have what we wanted: a self-contained DSO that defines malloc, and uses a different malloc for its own internal purposes, including for uses of malloc that occur transitively through other library calls. We have essentially got a “tailored deep copy” of the downstream library code, rather than the fragile monkey-linking we considered earlier. The flip side is we may have duplicated more code than was strictly necessary—remember we only set out to prevent reentrancy, not create something fully self-contained. • What shared objects really need to be aggressively self-contained like this? Dynamic linkers are one class. Clients of my libsystrap (https://github.com/stephenrkell/libsystrap/) are another—e.g.trace-syscalls-ld.so (https://github.com/stephenrkell/libsystrap/blob/615e3e293f677520c5ffbcfc217ff166f31c7aae/example/trace-syscalls.c) is an in-process system call tracer that catches all syscalls made by the host C library, and by any other code for that matter. It needs its own private non-trapping copy of any library routines, to avoid infinite regress. These come from a private copy of musl and a couple of other libraries. In other projects, the glibc dynamic linker itself is linked a similar way, against a private copy of glibc's routines. And my liballocs (https://github.com/stephenrkell/liballocs/) is also a client of libsystrap, and takes a similar approach again. (Though in contrast to the malloc use case, none of these DSOs is providing an interface that it also consumes—so they don't need to do the renaming trick.) • It may be surprising that we can simply link in a second copy of the C library, without the cooperation of the first one, and not break things horribly. Indeed, although this is commonly done, it's not 100% robust. It only works on the assumption that the code we pull in, being a subset of the whole libc, will only include code that is somehow “non-interfering”. By that I mean there are no common resources on which the separate copies would conflict. For example, to make malloc fit this description, we have to make it use mmap() not sbrk(). Each memory mapping exists independently of others, so there is no conflict, whereas there is only one brk pointer per process. Conflict on this could cause havoc, e.g.one malloc telling the OS to adjust brk downwards if it thinks everything up to the current break position has been freed, when the other malloc has independently allocated some chunks there without the first one knowing. • By getting into this sort of “resource conflict” analysis, we have essentially pushed the game one level down relative to our concern earlier. Earlier we were worrying about which libc calls might trigger reentrancy. Now we're instead trying to identify which calls might be hard-wired to static pieces of kernel state, like brk, or architectural state, like registers reserved for use by the thread-local storage implementation. The latter are another classic source of conflict with this sort of approach (and it really makes you glad that x86 provides both %fs and %gs; you can guess the switcheroo hack). I'd argue that getting involved in this “conflict analysis” is overall preferable to doing the reentrancy thing, because it's working at a narrower interface further down the stack, i.e.there's fewer of these “possibly conflictable” things than there are possibly-malloc-calling libc calls. After all that... in our Rust case I actually skipped the self-containment thing and opted for a simpler fix. The Rust library code that calls __cxa_thread_atexit_impl has a fallback path for when that symbol is not available. extern "C" { #[linkage = "extern_weak"] static __dso_handle: *mut u8; #[linkage = "extern_weak"] static __cxa_thread_atexit_impl: Option< extern "C" fn( unsafe extern "C" fn(*mut libc::c_void), • mut libc::c_void, • mut libc::c_void, ) -> c_int, ; } if let Some(f) = __cxa_thread_atexit_impl { unsafe { f( mem::transmute::< unsafe extern "C" fn(*mut u8), unsafe extern "C" fn(*mut libc::c_void), (dtor), t.cast(), core::ptr::addr_of!(__dso_handle) as *mut _, ); } return; } register_dtor_fallback(t, dtor); It probes for this using a weak reference, i.e.a symbol whose value defaults to zero if there is no definition provided at link time. To force the Rust code in our malloc library always to take the fallback path, we can take the unusual step of defining the __cxa_thread_atexit_impl symbol to zero, using a strong definition. That way, even if the surrounding link environment provides a definition for the function, ours will override it. To limit that override to just within our shared object. we also make our zero-valued definition hidden. Then our Rust code, still seeing the value zero for the __cxa_thread_atexit_impl symbol, will avoid using it and take the fallback path, just as if no definition were present in the process. Other objects in the process will be linked as normal and can make use of the global __cxa_thread_atexit_impl. • To nitpick here, there's really two orthogonal issues. We want our zero-valued definition to be hidden so that code in other libraries is not affected, and we also want our binding to it to be non-preemptible so that within our library, our hidden definition is always used even when there is another global definition (as there usually is). If we wanted just the non-preemptible thing we could use protected rather than hidden visibility for our definition. The phrase “protected visibility” in ELF is a misnomer: visibility of a protected symbol is not limited in any way, and in fact more things will bind to it than if it were not protected. You could think of it as the bindings being protected, not the symbol definition. Not being hidden, a protected definition can still be bound to from outside and can still preempt other libraries' definitions. The visibility should possibly be called “symbolic” in the sense of -Bsymbolic, since based on my non-bulletproof understanding, the effect of the latter is like giving every defined symbol protected visibility—although the word “symbolic” is hardly suggestive of these semantics either. Setting link options Both our solutions above involved a customised link job. How can we do that? Happily, rustc lets us supply “-C link-args=xyz” and it will pass xyz to the “link command”. Slightly irksomely, but I suppose only following Make's example, its notion of the “link command” is really not the linker but the C compiler driver, by default cc. So we first have to prepend -Wl, when we want to set a bona-fide linker option, and then prepend -C link-args= to all that. That's when talking to rustc, but MMTk's example “dummy VM” (https://github.com/mmtk/mmtk-core/tree/74dadfd112070f6b6cadca3df44d2c6375d742dc/docs/dummyvm) (the base for my malloc) uses Cargo for its build. So now we are three hops away from the actual linker: cc, rustc and then Cargo. How do we tell Cargo to pass “-C link-args=xyz”? I tried putting a line in Cargo.toml: rustflags = ["-C", "link-args=-Wl,-Bsymbolic"] ... under [lib] or under [profile.release] or any other profile. For me this doesn't work. It is possible to put it in .cargo/config.toml... [build.libmmtk_dummyvm] rustflags = ["-C", "link-arg=-Wl,-Bsymbolic"] ... but then it affects all rustc invocations during the build of the given target, even of dependencies. That's not what I want. I was told that a build.rs “build script” might provide a way around this. But I've also seen build.rs described (warning: neo-fascist ex-bird site) as a mistake (https://x.com/nick_r_cameron/status/1542771962350559232). To me it seemed like a needlessly big hammer. Instead we can simply ditch cargo build for cargo rustc. I even wrote a vaguely generic Makerule for doing this. (Yes, I am stubborn enough to use Make to run Cargo.) target/$(target)/lib%.so: cargo rustc -p $^ $(CARGO_RUSTC_FLAGS) \ • - $(RUSTC_FLAGS) \ $(foreach x,$(filter %.o,$+),-C link-args=$(x)) \ $(foreach x,$(LDFLAGS),-C link-args=$(x)) \ $(foreach x,$(LDLIBS),-C link-args=$(x)) Now, although with the addition of Make we might be four hops from the linker, we can set some linker flags like -Bsymbolic (if we want this) in the following joyously almost-direct way: target/$(target)/libmmtk_dummyvm.so: LDFLAGS += -Wl,-Bsymbolic and we can realise our __cxa_thread_atexit_impl by appending a linker script that defines our hidden symbol: $ grep . cxa-thread-atexit-dummy.lds PROVIDE_HIDDEN(__cxa_thread_atexit_impl = 0); ... and by adding in our Makefile: target/$(target)/libmmtk_dummyvm.so: cxa-thread-atexit-dummy.lds Note that I used a linker script because it's not possible to define a hidden symbol using only a linker command-line argument. If we didn't care about it being hidden, we could have used --defsym on the linker command line. As it is, we've effectively added a new input file to the link job, albeit a file that is only a linker script providing that hidden definition, rather than an object file defining input sections. Spoiler: at the time of writing, the above cxa-thread-atexit-dummy.lds script doesn't work! At least with the GNU linkers, it is specifically broken for shared objects, where the resulting symbol will take a non-zero value. See GNU binutils bug 32443 (https://sourceware.org/bugzilla/show_bug.cgi?id=32443) for what I think is happening. One workaround is to drop the hiddenness, e.g.by using PROVIDE rather than PROVIDE_HIDDEN, but that risks preempting the “real” definition of the symbol. Another workaround, although gross, is to use an IFUNC symbol whose value is computed as zero. That is what I currently do. Bundling some C or assembly code into the link We now have what we need to implement our malloc-family API in Rust, package it as a shared library and expose it to the wider process. But I actually set myself a more difficult plumbing problem: how do we create this library itself out of a mixture of C and Rust code? The core implementation is in Rust, but I'd like the option of a little bit of glue code in C... or maybe later in C++, or assembly, or Zig, or something else. One motivation is that a malloc isn't built in a day. Another is that, as we are finding, to work around various shortcomings of both Rust and binutils, we may need to add little bits of assembly or linker-script code to the link anyhow. And I do think this arises out of necessity not choice: I suspect it would not (currently) be viable to create a fully rustc-encapsulated solution, without giving any pass-through options to the linker. (At best, I don't know how to do it—tips welcome!) What glue code do I want to be able to link in? Well, I actually used MMTk only to define a single function: memalign. This is the one malloc-family function that suffices for my purposes. I then wanted to re-use some existing very simple glue code I already had that reimplemented malloc, calloc and so on in terms of memalign. It was articulated in a GitHub comment by Josh Triplett (https://github.com/mesonbuild/meson/pull/11213#issuecomment-1366541458) that a simple workflow for this use case, of linking in bits of non-Rust code, is definitely desired by the Rust project, but is “not there yet” (as of that comment, dated December 2022). But I'm naive, so let's try it anyway and see what breaks. We've already been adding stuff to the linker command. Why can't we just add some extra .o files to be linked in? We can, but unfortunately, their symbols won't be exported, so our malloc won't be usable. Why not? Well... $ cat hello.rs pub fn main () { println!("Hello, world!"); } $ rustc -C save-temps=yes --crate-type cdylib hello.rs -o /tmp/hello.so $ ls -ltd /tmp/rustc* | head -n1 drwxr-xr-x 2 stephen stephen 1024 Dec 11 05:01 /tmp/rustc4pJBIY $ ls -l /tmp/rustc4pJBIY total 2 • rw-r--r-- 1 stephen stephen 22 Dec 11 05:01 list • rw-r--r-- 1 stephen stephen 848 Dec 11 05:01 symbols.o $ cat /tmp/rustc4pJBIY/list { local: • ; }; When rustc links an ELF shared object, it passes a generated list file, like the one seen above, as the --version-script argument to the linker. The effect of the above script is that no symbols will be exported for dynamic linking. If #[no_mangle] is added to a function or variable defined in Rust code, it will be named as exported in the list, as an exception to the local star wildcard. (That's a little confusing, since mangling to me seems orthogonal to export. But I can understand how this might have come to be.) Since rustc is oblivious to our extra .o files, it obviously does not add their symbols to the list it generates. Owing to the local:* wildcard, anything to be exported needs to be named explicitly if we are providing such a list. Maybe we can override the wildcard by passing a second --version-script that names our extra symbols? Unfortunately neither the BFD linker nor gold can merge multiple version scripts, so this will error out. For the same reason, linker-level symbol versioning is currently not available from Rust (https://github.com/rust-lang/rust/issues/456) (and the issue closed!), even though it's still being mentioned (https://internals.rust-lang.org/t/support-symbolversioning-in-export-name/15021) as desirable (which it is). To bring this about, it would be necessary to forgo link-time optimization for now, since implementations of symbol versioning have only recently been reconciled with link-time optimization (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200), and in GCC but (as I last checked, a few months ago) not yet in LLVM. The issue seems to be that version information is traditionally added by assembly directives dropped into C or C++ source without the compiler's knowledge, using a top-level asm block. It therefore does not survive the link-time IR reprocessing. • Although the orthodox approach to solving this is by adding compiler-level features for symbol versioning, I personally would say instead that the IR-bundling approach to link-time optimization (LTO) is the problem. In my view, this approach to LTO is a giant hack that is fundamentally flawed, in that it removes the object code abstraction from the programmer. Non-trivial amounts of programmer intent—including symbol versioning, but not only that—have traditionally been expressed at that level; we shouldn't be forced to lose it! The IR-bundling approach also creates footguns when debugging build or optimisation problems, because it puts into circulation a version of the object code that will be thrown away or ignored. My view is that LTO should work instead by lifting the bona fide contents of .o files back up to IR where that is needed—using debugging information, suitably extended, to recover any useful semantic information that may have got lost. Of course, IR itself is a valid representation for bona-fide .o contents. Still I'd defend the feasibility and practicality of lifting back up from target machine to IR where necessary. This is because the same same “useful semantic information” needed for high-quality lifting would also be useful for high-quality debugging. (This could be good topic for somebody's PhD... do get in touch!) Disabling LTO isn't a complete showshopper of course. The more pressing issue is also the more mundane: the Rust compiler has co-opted the version script feature in order to control its own export namespace. If you actually want to version symbols—you know, the thing that feature was really intended for—you'd have to add a second version script, but as we know, that happens not to work. Actually I'm lying slightly: it does work with lld, which can merge multiple version scripts. So the workaround (https://users.rust-lang.org/t/how-to-use-linker-version-scripts-in-rust-1-54/64033) is to use that. (As the linked-to page reveals, just doing that is not quite enough: we also need to add a separate --defsym versionable alias for each versioned symbol defined in Rust.) Assuming we're not using lld, is it necessary to spend the precious unique --version-script for this symbol-hiding purpose? This is where things get very murky. Another linker option is morally a cut-down version of --version-script, called --dynamic-list, and exists primarily to control the export of symbols for dynamic linking. It seems like exactly what we need, and it can be specified multiple times on the command line. Can we just rewrite rustc's use of --version-script into the equivalent use of --dynamic-list? Unfortunately not! There are two problems. The first is that while --dynamic-list lets us add exportedness (in some sense), it doesn't let us take it away, which is what rustc wants to do. Secondly, and rather insanely, --dynamic-list only lets us add exportedness when producing executables. When producing shared libraries, it was redefined to control instead the preemptibility of dynamic symbols, not their exportedness; it doesn't affect what goes in the dynamic symbol table. To take away exportedness, there is another option --exclude-libs... but it only affects symbols pulled in from archives. There's no way to ask the linker to remove visibility from symbols defined in .o files. The only way is to remove the visibility on the input side, in the .o files themselves, whether using compiler options like -fvisibility=hidden or just setting the visibility directly per symbol. • I played with some contortions for having --exclude-libs take effect for all our link inputs. In particular we can bundle all the .o files into an archive and then link it with --whole-archive, which takes away most archivey semantics but keeps --exclude-libs. Still, although that lets us remove exportedness, in the case of shared library output we remain stuck in that we can't use --dynamic-list to selectively add it back. No matter; there's another similar option in recent GNU linkers: --export-dynamic-symbol-list. Surely that will do what we want? Sadly, again not when making shared libraries! The patch that introduced it (https://inbox.sourceware.org/binutils/20200523182928.dwqe2dmtyupxzrpw@google.com/T/) said “it is treated like --dynamic-list if -Bsymbolic or --dynamic-list are used, otherwise, it is ignored, so that [i.e.either way,...] references to matched symbols will not be [pre-]bound to the definitions within the shared library”. So again it is defined to control preemption but not visibility or exportedness. The only difference from --dynamic-list is that it doesn't implicitly turn on any aspect of -Bsymbolic. The pithy-ish summary: “references to matched non-local STV_DEFAULT symbols shouldn't be bound to definitions within the shared object even if they would otherwise be due to -Bsymbolic, -Bsymbolic-functions or --dynamic-list”. Is it intentional that these options cannot be used to force a symbol into the dynamic symbol table of a shared object, and therefore have no effect in respect of --exclude-libs? I think it is not. Indeed it's recognised as a bug (https://sourceware.org/bugzilla/show_bug.cgi?id=16693) at least in the GNU gold linker. But that bug was also closed as WONTFIX, with the rather disappointing rationale (abbreviating wildly) “just use --version-script”. We are back to square one! As a stopgap, it's time for a big hammer. If our link command is a cc that is really gcc, we can use the -wrapper option to interpose on the linker command. We can delete the version script, rewrite it, or make whatever other changes we please. For now I've chosen simply to remove it and take the responsibility for hiding symbols myself, printing a warning. Annoyingly, rustc will gobble this stderr output unless we make an arcane RUSTC_LOG incantation. So here's how I do a trivial link of a hello.so cdylib. $ RUSTC_LOG=rustc_codegen_ssa::back::link=info \ rustc -C save-temps=yes --crate-type cdylib hello.rs -o /tmp/hello.so \ • C link-args=-wrapper -C link-args=mywrapper Warning: dropping rustc-generated version script /tmp/rustcGDnffy/list: { global: main; local: • ; }; • This decision to gobble the linker stderr output is one more thing betraying a recurring design choice in the Rust toolchain: to hide the linker from the user, to a much greater extent than C or C++ compilers do (which is still non-zero, of course). The theme of this post is that non-routine linking operations are variously buried or inaccessible in Rust, currently. Opinions will differ, but in the case of a self-identifying “systems programming language”, I regard hiding the linker as a mistake. I could rant much more about this. How can we go about hiding symbols ourselves afterwards? It's the sort of thing that can be done with tools like objcopy. But again, not quite! I wrote “tools like objcopy” not objcopy itself, because that (at least the GNU binutils version) does not grok the dynamic symbol table. I do have some tools in my elftin (https://github.com/stephenrkell/elftin/) repository that can manipulate the dynamic symbol table in a few ways; not the way we need right now, currently, but adding one more is not that hard. (Caveat: I haven't done it yet, and it would involve regenerating the dynamic symbol hash tables, which is not entirely trivial.) For now I can simply maintain a hand-crafted version script. The only difference with the rustc-generated one is that it adds the malloc-family functions to the global list. • A more useful compiler wrapper might add support for combining multiple version scripts into one, as a workaround for those linkers that don't support that. That's too much syntax for me right now, but it's pretty tractable. In general, the wrapper script feature is a nice point of interposition for extending the compiler in some controlled black-box ways. It has its foibles, which I've blogged about at length (https://www.humprog.org/~stephen/blog/2025/05/02/site_root_url/blog/2024/08/27#how-to-wrap-cc-really) recently, of course. Questions left to answer Forcibly dropping rustc's version script is an ugly thing to do. How did we get here? Our most immediate problem was that --exclude-libs is too blunt a hammer because we can't selectively undo it. But popping up one level, rustc is telling the linker the selection of symbols that it thinks should be global and also that all others should be local (or, mostly-equivalently, hidden). Currently it assumes that “all Rust input objects” is the same as “all input objects”, hence its use of “local:* in the version script. This is overreach in our situation. And I'd argue this is really just a case of rustc being coded a little “too expediently” at present. Ultimately, it can enumerate its own symbols! If we dig into the compiler code, we know that rustc already has an explicit list not only of which symbols it wants to export but also which it doesn't. In other words, the wildcard is unnecessary. What I think everyone wants is for the hiddenness be limited to the Rust objects of the link. Why not just enumerate those symbols in the generated script? Or better still, why not ditch the script entirely and generate the right symbol visibilities in the linker inputs as they emerge from rustc? So I suspect that is the right solution. I've dug some way into this, but not right to the bottom— hopefully in a future post I will get there and have a patch to propose to rustc. Popping back to malloc With our big-hammer solution in place, we can link a shared object from a mix of C and Rust code. The result is a happily preloadable malloc that the rest of the system can use. And when we combine the preloading with liballocs, we can introspect on its allocations just like with the system malloc. Currently we have to keep an ad-hoc version script or else export too many Rust symbols, but I can live with that for now. The next things I want to do with MMTk are to use its VO bits (“valid objects”) feature for implementing the lookup of object base addresses, as a replacement for (or perhaps generalisation of) liballocs's existing “generic malloc” indexing code. Doing that should be only a small delta from current liballocs, since this code already uses a bitmap. But it will be a step closer to a liballocs that can work with the wider (open) family of MMTk-based allocators. It would also be nice to a simple free list and free implementation, although it appears that (not surprisingly) explicit freeing is not a clearly surfaced feature in MMTk. Another way to think of MMTk is a “heap implementation generator”—the code is highly parameterised and uses Rust's generics pervasively. So, every compiled “instance” of MMTk looks a little bit different, not only on its implementation but also its interfaces as they emerge from the compiler. Another integration task, therefore, is to automagically wrap this family of interfaces in a “dynamic dispatch” layer that can plug any of these generated heaps into liballocs—initially to allow queryability, but maybe also to support its uniform base-level API too, for allocation operations proper. Probably this could happen in some generative Rust magic; achieving that is going to be an education for me. You can find my code here (https://github.com/stephenrkell/mmtk-liballocs), in the ‘malloc-trivial’ subdirectory. I'll put other MMTk–liballocs integration experiments in the same repository as I go along. That's all for now, save an obligatory reminder of [the bit up top (https://www.humprog.org/~stephen/blog/2025/05/02/#replacing-malloc-using-mmtk-in-rust). :-)] [/devel (https://www.humprog.org/%7Estephen/blog/devel)] [all entries (https://www.humprog.org/%7Estephen/blog-all.html)] permalink (https://www.humprog.org/%7Estephen/blog/2025/05/02#replacing-malloc-using-mmtk-in-rust)contact (https://www.humprog.org/~stephen/#contact) • http://blosxom.sourceforge.net/ validate this page (http://validator.w3.org/check?uri=https://www.humprog.org/%7Estephen/blog)