Why am I writing a Rust compiler in C?
(notgull.net)
from thomask@lemmy.sdf.org to rust@programming.dev on 25 Aug 2024 21:01
https://lemmy.sdf.org/post/21429534
from thomask@lemmy.sdf.org to rust@programming.dev on 25 Aug 2024 21:01
https://lemmy.sdf.org/post/21429534
threaded - newest
This project sounds kind of masochistic but the idea is to bootstrap Rust from tinycc, and have traceability down to the lowest level assembly code. There is a step missing though? Tinycc is written in C after all.
I think it would make more sense to bootstrap from a small Lisp written in assembly language, if the traceability goal is worthwhile at all. There is nothing special about C.
I wish that were true, but isn’t it somewhat wishful thinking? Even an assembly-language Lisp would require an operating system in order to build a functioning compiler, wouldn’t it? And operating system APIs are in C.
Edit: more importantly, as the post explains, the special thing about C is the existence of TinyCC.
In principle you could start from hand assembly. Look up “sectorlisp” as a lowest level option. Or you could start from Forth, which is traditionally implemented using very simple methods. The blog post really doesn’t make clear what problem the author is trying to solve. It gives some general description but leaves a lot to be guessed at.
Then there is the question of where the CPU is supposed to come from. Any modern one was designed using lots of mysterious CAD tools. Maybe scrounge a vintage Z80 out of an old Timex-Sinclair or something?
The author doesn’t explain exactly what their interest in bootstrapping is, but the goal is pretty explicit:
The deeper reasoning is still not explained. C is just terrible for this. Rust is very complicated and writing a new implementation is a big project even in good languages. So using C seems tragic.
I’m going to go further and say that TinyCC isn’t bootstrapped either, since the compiler writer’s thought processes aren’t bootstrapped. You would have to use something like CompCert (i.e. all the reasoning that the programs works is embedded in the program and machine checked) and bootstrap that. It is probably doable, but not as a 1 person hobby project.
Okay. That’s just imposing a different (and at least equally arbitrary, if not moreso) definition of bootstrapping. Why does it matter that the author didn’t explain their “deeper reasoning” for having an interest in bootstrapping, or the Bootstrappable Builds project specifically? If you feel like that project isn’t meeting a sufficient bar for what counts as “bootstrapping”, or that using C as the first high-level language they bootstrap is “tragic”, take it up with that project, I guess.
I will take a look at the bootstrapping project page, but “bootstrappability” is a philosophical notion whose extent depends on what you are trying to get from it. Certainly someone who pursues it should give that some thought and reach a conclusion, rather than just following a recipe on some web site. So that’s the deeper reasoning I felt was missing.
As for C being terrible, well, why would I want to take that up with anyone? It’s simply that we know from 50 years of experience with C that writing bug-free C programs, or noticing the existence of bugs in them, is extremely difficult. If someone decides to use it for bootstrapability anyway, xkcd.com/386 would seem to apply.
collapseos.org (which uses Forth) might also be of some interest, though I think that was another questionable decision. Real transparency and boostrapability requires that the reasoning process be written out and matched up with the code. C does a pretty poor job of that compared to some alternatives.
It seems it would be a lot easier to work on the GCC compiler, and work with others to bootstrap GCC (if it hasn’t already been done). Getting the GCC Rust compiler able to compile some version of rustc probably wouldn’t be that hard, and then you can just use that version to compile up the chain to modern Rust.
That’s the
mrustcproject the author mentions. He wants Rust to be bootstrapped earlier in the process.Sure, but I guess I don’t really understand the argument. Why would Rust need to be involved earlier in the process? Isn’t the point to have a way to compile rustc completely from source?
I guess it’s cool to have multiple ways to get there, but that project would take way less work and get to the same end goal. It sounds to me like the author is trying to justify a cool project instead of trying to solve a real problem. That’s completely fine, but I think most people would be happy with the mrustc project.
I asked, and it’s to replace some of the bits that require Perl: hachyderm.io/@notgull/113035157972265244
You can see the full (current) sequence here: bootstrapping.miraheze.org/wiki/Live-bootstrap
Ben Eater has entered the chat
Operating system interfaces use the C ABI, but they don’t require you to use C.
Actually that’s true on most OSes, but not Linux - that uses syscalls as its interface not function calls to a shared library, so it’s the syscall ABI.
I still feel like designing and bootstrapping your own higher level language is going to be less painful overall than writing a Rust compiler in C. And probably more fun.
The main thing is that TinyCC has already been bootstrapped.
Check out this page on bootstrappable.org. Basically they start with a 200 something byte binary (hex0) that can act as an assembler, then using a bunch of layers of tools and compilers you can bootstrap a whole system. I think they use stage0 to build M2-Planet, use that to build GNU Mes, and use that to build TinyCC.
So a project like this fits neatly into that bootstrapping path. It could be done other ways, but starting from a fairly complete C compiler makes it a lot easier than building an entire path from scratch.
Didn’t Rust itself used to have a compiler written in a different language? I mean, obviously it would’ve had to at some point but I think I remember reading about them abandoning it once they didn’t need it. Why not use that?
You can technically do it, but it’s a convoluted path. The article talks about it. Basically to bootstrap that way you need to go through a lot of versions of rust, compile rust 0.7 in ocaml, compile ocaml in scheme, and compile scheme in C using gcc. For gcc you need to compile a chain of versions back to when it was written in C instead of C++, plus the whole TinyCC bootstrapping path.
edit: had listed scala instead of ocaml
Ah okay. The article was a little over my head so I mostly skimmed it. This makes sense what you’re saying though. It’s easy to forget the level of bootstrapping they’re trying to do is all the way to assembly.
It’s the sort of thing if you think about too long you’ll get paranoid and start using Gentoo exclusively lol.
Not sure if there was another rewrite, but AFAIK (and the article agrees with me) rustc was originally written in Ocaml
Yeah, I wrote the wrong language. I tend to lump those together in my head as ‘big multi-paradigm languages I haven’t bothered to learn yet.’
If you had read the article, you’d know!
I’m still lost on why they’re doing it.
A lot of this bootstrapping stuff comes back to the ‘trusting trust’ attack. You could write a compiler that adds some malicious code to programs it compiles. But the thing is, it could also insert it’s own malicious code when compiling a compiler. So you look at your code, and the code of your compiler, and everything looks fine, but the exploit is still there. Someone wrote an example in rust.
Theoretically there could also be bugs that propagate this way. So to fully trust your code is doing what you think it is, you’d need a chain of compilers back to hand coded assembly.
Why would a Rust compiler written in C be more trustworthy than one written in Rust?
If the idea is that, in an ideal world, we would compile each layer of compilers from assembly-up-to-Rust for each build, that seems even more risky as then you have to trust each compiler instead of just one.
That’s already how it is now, we just don’t usually think of it that way. You can’t compile rust unless you already have a rust compiler. The current version was compiled in a previous version, which was compiled in a previous version, going through a chain of older versions and other languages. Anything along that chain could’ve theoretically had an influence on the current compiler.
It’s not about the code itself being more trustworthy. The point is that when you bootstrap, you don’t have to blindly trust any of the binaries, since it’s source code the whole way down. Someone could bootstrap rustc like this, compare it to the binaries that already exist, and ideally they would be identical.
Oh, okay. Thank you for clarifying. So doesn’t that mean we should never have a compiler written in the same language that it compiles? Why would we ever choose to make the mistake of using the same language? Is it ever not a mistake?
It makes sense that if you’re designing a language, you’d like the language you made and would want to use it. It’s fine for compilers like that to exist, and even be the main one used, but ideally it shouldn’t be the only compiler.
But there are technically ways to bootstrap a language without writing it in another language (other than a small core in assembly or something). You could design a tiny compiler that only handles a small subset of your language, then write a better compiler using only the features available in that subset. You can do this for several layers of compilers until you have the full language.
I wonder if someone’s working on this from the other end. Like, instead of porting mrust from gcc to tinycc, port rust 1.79 from 1.78 to 1.1.