Why Rust for Low-level Linux programming?

I think Rust is extremely well-suited for low level Linux systems userspace programming — daemons, services, command-line tools, that sort of thing.

Low-level userspace code on Linux is almost universally written in C — until one gets to a certain point where it’s acceptable for Python to be used. Undoubtedly this springs from Linux’s GNU & Unix heritage, but there are also many recent and Linux-specific pieces that are written in C. I think Rust is a better choice for new projects, and here’s why.

Coding is challenging because of mental context-keeping

Coding is hard and distractions are bad because of how much context the developer needs to keep straight as they look at the code. Buffers allocated, locks taken, local variables — these all create little mental things that I need to remember if I’m going to understand a chunk of code, and fix or improve it. Why have proper indentation? Because it helps us keep things straight in our heads. Why keep functions short? Same reason.

Rust reduces the amount of state I need to keep track of in my brain. It checks things that before I depended on myself to check. It gives me tools to express what I want in fewer lines, but still allows maximum control when needed. The same functionality with less code, and checks to ensure it’s better code, these make me more productive and introduce fewer bugs.

Strong types help the compiler help you

Strong typing gives the compiler information it can use to spot errors. This is important as a program grows from a toy into a useful thing. Assumptions within the code change and strong typing check the assumptions so that each version of the program globally either uses the old, or the new assumptions, but not both.

The key to this is being able to describe to the compiler the intended constraints of our code as clearly as possible.

Expressive types prevent needing type “escape hatches”

One problem with weakly-typed languages is when you need to do something that the type system doesn’t quite let you describe. This leads to needing to use the language’s escape hatches, the “do what I mean!” outs, like casting, that let you do what you need to do, but also inherently create places where the compiler can’t help check things.

Or, there may be ambiguity because a type serves two purposes, depending on the context. One example would be returning a pointer. Is NULL a “good” return value, or is it an error? The programmer needs to know based upon external information (docs), and the compiler can’t help by checking the return value is used properly. Or, say a function returns int. Usually a negative value is an error, but not always. And, negative values are nonzero so they evaluate as true in a conditional statement! It’s just…loose.

Rust distinguishes between valid and error results much more explicitly. It has a richer type system with sum types like Option and Result. These eliminate using a single value for both error and success return cases, and let the compiler help the programmer get it right. A richer type system lets us avoid needing escapes.

Memory Safety, Lifetimes, and the Borrow Checker

For me, this is another case where Rust is enabling the verification of something that C programmers learned painfully how to do right — or else. In C I’ve had functions that “borrowed” a pointer versus ones that “took ownership”, but this was not enforced in the language, only documented in the occasional comment above the function that it had one behavior or the other. So for me it was like “duh”, yeah, we notate this, have terms that express what’s happening, and the compiler can check it. Having to use ref-counting or garbage collection is great but for most cases it’s not strictly needed. And if we do need it, it’s available.

Cargo and Libraries

Cargo makes using libraries easy. Easy libraries mean your program can focus more on doing its thing, and in turn make it easier for others to use what you provide. Efficient use of libraries reduce duplicated work and keep lines of code down.

Functional Code, Functional thinking

I like iterators and methods like map, filter, zip, and chain because they make it easier to break down what I’m doing to a sequence into easier to understand fundamental steps, and also make it easier for other coders to understand the code’s intent.

Rewrite everything?

It’s starting to be a cliche. Let’s rewrite everything in Rust! OpenSSL, Tor, the kernel, the browser. Wheeee! Of course this isn’t realistic, but why do people exposed to Rust keep thinking things would be better off in Rust?

I think for two reasons, each from a different group. First, from coders coming from Python, Ruby, and JavaScript, Rust offers some of the higher-level conveniences and productivity they expect. They’re familiar with Rust development model and its Cargo-based, GitHub-powered ecosystem. Types and the borrow checker are a learning curve for them, but the result is blazingly fast code, and the ability to do systems-level things, like calling ioctls, in which a C extension would’ve been called for — but these people don’t want to learn C. These people might call for a rewrite in Rust because it brings that component into the realm of things they can hack on.

Second, there are people like me, people working in C and Python on Linux systems-level stuff — the “plumbing”, who are frustrated with low productivity. C and Python have diametrically-opposed advantages and disadvantages. C is fast to run but slow to write, and hard to write securely. Python is more productive but too slow and RAM-hungry for something running all the time, on every system. We must deal with getting C components to talk to Python components all the time, and it isn’t fun. Rust is the first language that gives a system programmer performance and productivity. These people might see Rust as a chance to increase security, to increase their own productivity, to never have to touch libtool/autoconf ever again, and to solve the C/Python dilemma with a one language solution.

Incremental Evolution of the Linux Platform

Fun to think about for a coder, but then, what, now we have C, Python, AND Rust that need to interact? “If only everything were Rust… and it would be so easy… how hard could it be?” 🙂 Even in Rust, a huge, not-terribly-fun task. I think Rust has great promise, but success lies in incremental evolution of the Linux platform. We’ve seen service consolidation in systemd and the idea of Linux as a platform distinct from Unix. We have a very useful language-agnostic IPC mechanism — DBus — that gives us more freedom to link things written in new languages. I’m hopeful Rust can find places it can be useful as Linux gains new capabilities, and then perhaps converting existing components may happen as maintainers gain exposure and experience with Rust, and recognize its virtues.

The Future

Rust is not standing still. Recent developments like native debugging support in GDB, and the ongoing MIR work, show that Rust will become even better over time. But don’t wait. Rust can be used to rapidly develop high-quality programs today. Learning Rust can benefit you now, and also yield dividends as Rust and its ecosystem continue to improve.

33 thoughts on “Why Rust for Low-level Linux programming?

  1. I agree with a lot of this, but a discussion needs to be had. Thing is, I believe for Rust to be useful for this, we need to have a local shared object dylib in the system. Otherwise each executable would have to be statically linked. This uses a lot of disk space.

    “So what!?” Asks the person with the 2TB HDD, but the person running Linux on 4GB of onboard flash on an SoC (System on a Chip) or even harder with 8MiB of onboard flash on a tiny MIPS system and the root pushed into a 4MiB SquashFS partition will want something that doesn’t eat 400+kiB per executable.

    This is something we should figure out so that we can redistribute these without a huge tax on the system.

    Like

  2. So happy to see this post. Glad you’re enjoying Rust 🙂

    Anders, Rust supports dynamic linking today. It’s just not the default. We didn’t make it the default because we don’t have a stable ABI. But if a distro wanted to ship a compiler, and packages are happening, they could dynamically link just fine, as everything would be using the same compiler version.

    Like

  3. Beyond the userspace, what are the prospects of using Rust deeper in the kernel? I’m not an expert in either C or Rust, but Rust’s safety features sound like they’d go a long way to avoid common C bugs and security vulnerabilities in the kernel.

    Like

  4. Steve,
    Yes, I know it has dynamic, it’s just that most current distros don’t ship with it, and past solutions for things like GNUStep have seemed less-than-optimal to me.
    I just think some talking about it to work out a deployment standard might be very helpful to the distros in the future.

    Like

  5. Anders, makes a lot of sense. I know that we care a lot about getting Rust into distro packages, and have done a bunch of work to make it easier, as an example, starting with Rust 1.10, we bootstrap from the previous release, rather than a snapshot. This is really important for distros, so that they don’t need a separate bootstrapping package. http://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767 is one of the places we’re trying to work on this, and solicit feedback.

    I also expect that as Firefox’s usage of Rust grows, that will also help. Firefox 45 already has Rust in it on Linux, though only a small bit. “build from source” distros will eventually want a rustc package to be able to build their Firefox.

    Like

  6. I think I’m going to write my next project in rust, out of curiosity though is it straightforward to implement IPC rust to c?

    Like

  7. Rust is the first language that gives a system programmer performance and productivity

    Well, that’s not true. C++ gave it to us a long before. Maybe not C++ alone, but Qt/C++ and Boost/C++. It’s also getting pretty nice to code with since C++11.

    Like

  8. I think the memory safety alone is reason enough to write new system level software in rust instead of C, and even to rewrite Old ones.

    Given the huge portion of security bug and CVE related to memmory access problems that cpuld not happen in safe rust code.

    Yes it will be a lot of work, but the security gains would outweight the cost alone.
    And then there are all the other arguments you cited.

    Like

  9. joeskb7, that’s a fair comment. But compared to C++, Rust is a much simpler language. Getting productive in a C++ project requires you to learn which subset of C++ that project uses.

    Of course, the really big advantage of Rust over C++ is its much stronger safety invariants.

    Like

  10. Rust aborts on OOM. That’s a complete deal breaker for a lot of embedded and systems programming use cases. The situation around dynamic linking is also totally unusable to the point where the support for it may just as well not exist. The Rust devs and fanboys are so deep in their own little world that they cannot even see from another perspective than their own even if they try.

    Like

  11. @Rangvald: there is dnf around, yum replacement on top of libzypp, recently made available for el7 as well (in EPEL). It’s blazing fast. 🙂

    Like

  12. I wanted to ask how Vala compares to Rust when it comes to writing low-level programming? What are the pros and cons of one over the another?

    Like

  13. So how does the performance of Rust? Is it equally fast or faster? Or is it a bit slower? If yes how much slower?

    Like

  14. Some time ago I got interested and tried the “Hello world!” example.

    Source code size: 45 B
    Compiled executable: 759 KB

    I stopped immediatelly.

    Like

  15. [quote]
    Rust is the first language that gives a system programmer performance and productivity.[/quote]
    I think D also does a good job in that space. It includes a GC and therefore has a certain performance penality, but overall it’s a good mixture of high-level features and low-level performance. You get a bit more programmer productivity than Rust but pay with (potentially) reduced performance

    And I say this as someone who really likes Rust.

    Like

  16. @John C

    Rust aborts on OOM.

    The rust standard library aborts on OOM, which is a reasonable enough decision for application code. Rust itself has no requirement to abort on OOM, and there’s no requirement to use the stdlib. The core library has no allocation, and people work on top of that to build libraries suitable for embedded/kernel programming.

    @Ahmed

    Rust statically links stdlib and jemalloc by default, so executables for small code packages indeed appear overlarge. As your code size grows you’ll find that your executables appear much more reasonable in size. Alternatively, there’s no requirement to statically link.

    Like

  17. @Ahmed: A hello world binary in Rust is not 759KB. It’s more along the lines of 5KB if you use dynamic linking and strip the binary. If you do not use dynamic linking, jemalloc and the Rust standard library is statically linked into every binary unless you choose to use the system allocator, or you create your own custom allocator. Using LTO can greatly reduce the final size as well for static linking.

    Like

  18. What about license incompatibility? There is no GPL version of Rust and we already had issues with questionable licenses in the past.

    Like

  19. I totally agree that most new code should be written in Rust rather than C or C++; and most of the reasons mentioned in this article are sound — aside from the bit about Autoconf.

    It is true that for straightforward cases, Cargo by now should indeed be able to cover most of what Automake+Libtool would be needed for otherwise. There is however nothing inherent in Rust or Cargo that avoids the need for Autoconf. Rust projects only get away without Autoconf because they simply ignore the problems Autoconf solves. This is fine for things like web services, that have no build-time options; and that are meant to run only in a specific environment, and/or work at a level where differences in operating systems and other aspects of the environment are mostly irrelevant. However, for Rust/Cargo to actually become a viable choice for portable low-level software, these problems will need to be addressed.

    This is painfully visible in the Rust compiler itself (or more specifically, the standard library), which is surely the most portable Rust project — and to achieve that, requires an ugly mess of system-specific build instructions and endless #[cfg] case handling. This is way worse than any (properly set up) use of Autoconf.

    Even worse is handling of external dependencies. Missing dependencies aren’t diagnosed up front, but rather simply result in obscure build failures. What’s more, complex projects such as Servo have unconditional dependencies on many libraries that should be optional, simply because it’s tricky to make these things optional with Cargo. And even if build-time options are implemented, it’s still hard to actually make use of them: with no auto-detection, and no straightforward way to list available options and their meaning.

    Not to mention build-time configuration of things that not just binary options… I don’t think this is currently possible with Cargo alone at all.

    These are all things that Autoconf takes care of, and that Rust/Cargo will somehow have to handle as well. One way is to actually use Autoconf with Cargo, as demonstrated at http://aravindavk.in/blog/autoconf-for-rust-projects/

    Like

  20. I strongly disagree with every subsection of this blog post. Point by point:

    “Coding is challenging because of mental context-keeping” — only applies to the beginner, and to those for whom study is akin to learning to recite the phone book. In actuality buffers allocated, locks taken, and the meaning of local variables is like concepts that’re referred to as “it”, “them”, “those”, “former”, “latter”, and so forth in English: foundational primitives which, if the programmer doesn’t understand, s/he should consider a different career.

    “Strong types help the compiler help you” — except when “strong types”, in the sense of something beyond C (which when written to the standard is quite strongly typed), amount only to the programmer having to hand-hold the compiler. See the point about the “borrow checker”.

    “Expressive types prevent needing type ”escape hatches”” — this doesn’t stop Rust from having an escape hatch nonetheless. This implies that Rust’s type system is not “expressive” enough (whatever that word is supposed to mean): code beyond the standard library is expected to fall back to unsafeness when the going gets rough enough. So far all attempts to model a program through the type system have either collapsed under the weight of their own impossibility, or resulted in programs that’re too rigid to be modified without a full rewrite. What use is such a methodology, when even slightly flexible languages (Ada, for an example of traditional BDSM) address changing requirements in stride?

    “Memory Safety, Lifetimes, and the Borrow Checker” — on the contrary, the borrow checker is the worst part of Rust. Its major upshot is Rust’s move requirements: the programmer is required to keep track of data lifetime within any given scope, even between individual variables. If s/he does not, the compiler gives an ugly error message. No other language disallows use of a variable after its value has been assigned to a different variable in the same scope! Worse still, in order for a function to access a chunk of data that’s expected to remain afterward, the program must pass its sole reference to the function and then receive it back out again, if such usage wasn’t foreseen with a reference-counted cell type — in which case the language requires refcount bumping at both call and return.

    “Functional Code, Functional thinking” — however, unlike civilized functional programming languages like Common Lisp, and imperative languages with functional primitives like Perl and Ruby, side effects within closures are disallowed. Furthermore, if ref-cell types are closed over, the closure can only be invoked exactly once — because invocation causes destruction of its allocated context. This is objectively less useful than the C style of closure, being a function pointer and an userptr; and in all ways worse than any of the higher-order primitives Haskell provides, such as mapAccum[LR], mapConcat, and the various folds, or Lisp’s equivalents which explicitly allow for bidirectional side effects from within lambdas.

    “Rewrite everything?” — I strongly recommend this as an exercise for anyone naïve enough to believe in Rust. Indeed doing so would go quite a ways to proving Rust a mature language, which it so far most definitely is not. Ideally the Rust version should be maintained concurrently with whatever it seeks to replace, in order to further prove that contrary to evil tongues (such as mine here) Rust doesn’t handcuff the programmer to an initial design.

    “Incremental Evolution of the Linux Platform” — this is what they said about C++, Java, O’Caml, C#, and all those other languages that appeared, were fluffed up for a few years, and then went back to their respective niches. For example, Corel was bankrupted by their attempt to rewrite everything in Java; many a project has been sunk by C++/C# style overdesign; and too many programs never came to be because their authors started out in O’Caml without having a Perfect Data Design up front and so painted themselves into a corner.

    Like

  21. @antrik

    Rust has no ABI stability. That completely rules it out for one of the use cases that it’s potentially best suited for (dynamic libraries shipped as distro packages).

    Like

Leave a comment