Tag Archives: rust

Stratis 1.0 released!

5 Replies

We just tagged Stratis 1.0.

I can’t believe I haven’t blogged about Stratis before, although I’ve written in other places about it. We’ve been working on it for two years.

Basically, it’s a fancy manager of device-mapper and XFS configuration, to provide a similar experience as ZFS and Btrfs, but completely different under the hood.

Four things that took the most development time (so far)
  1. Writing the design doc. Early on, much of the work was convincing people the approach we wanted was a good one. We spent a lot of time discussing details among ourselves and winning over internal stakeholders (or not), but most of all, showing that we had given serious thought to various alternatives, and had spent some time to comprehend the consequences of initial design choices. Having the design doc made these discussions easier, and solicited feedback that resulted in a much better design than what we started with.
  2. Implementing on-disk metadata formats and algorithms to protect maximally against corruption and over-write. People said it would take more time than we thought and they…weren’t wrong! I still think implementing this was the right call, however.
  3. The hordes of range lists Stratis manages internally. It was probably inevitable that using multiple device-mapper layers involves a lot of range mapping. Stratis does a lot of it now, and it will be doing way more in the future, once we start using DM devices like integrity, raid, and compression. Rust really came through for us here I think. Rust’s functional aspects work very well for things like mapping and allocating.
  4. The D-Bus interface was a big effort in the pre-0.5 timeframe, but now that it is up and running it’s easy to maintain and update. We owe much of this to the quality of the dbus-rs library, and the receptivity of its author, diwic, to help us understand how to use it, and also helping to add small bits that aided our usage of D-Bus.
People to thank

Thanks to Igor Gnatenko and Josh Stone, two people who played a large part in making Rust on Fedora a reality. When I started writing the prototype for Stratis, this was a big question mark! I just hoped that the value of Rust would ensure that sooner or later Rust would be supported on Fedora and RHEL, and thanks to these two (and others, and, oh, you know, Firefox needing it…) it worked out.

I’d also like to thank the Rust community, for making such a compelling, productive systems language through friendliness and respect, sweating the details, and sharing! Like I alluded to before, Rust’s functional style was a good match for our problem space, and Rust’s intense focus on error handling also was perfect for a critical piece of software like stratisd, where what to do about errors is the most important part of what it does.

Finally, I’d like to thank the other members of the Stratis core team: Todd, Mulhern, and Tony. Stratis 1.0 is immeasurably better because of the different backgrounds and strengths we each brought to bear on developing this new piece of software. Thanks, everybody. You made 1.0 happen.

The Future

The 1.0 release marks the end of the beginning, so to speak. We just left the Shire, Frodo! Stratis is a viable product, but there’s so much more to do. Integrating more high-value device-mapper layers, more integration with other storage APIs (both “above” and “below”), more flexibility around adding and removing storage devices, while keeping the UI clean and the admin work low, is the challenge.

Stratis is going to need some major help to get there. For people interested in doing development, testing, packaging, or using Stratis, I invite you to visit our website and GitHub, or just keep tabs by following the project on Google Plus or Twitter.

Why Rust for Low-level Linux programming?

33 Replies

I think Rust is extremely well-suited for low level Linux systems userspace programming — daemons, services, command-line tools, that sort of thing.

Low-level userspace code on Linux is almost universally written in C — until one gets to a certain point where it’s acceptable for Python to be used. Undoubtedly this springs from Linux’s GNU & Unix heritage, but there are also many recent and Linux-specific pieces that are written in C. I think Rust is a better choice for new projects, and here’s why.

Coding is challenging because of mental context-keeping

Coding is hard and distractions are bad because of how much context the developer needs to keep straight as they look at the code. Buffers allocated, locks taken, local variables — these all create little mental things that I need to remember if I’m going to understand a chunk of code, and fix or improve it. Why have proper indentation? Because it helps us keep things straight in our heads. Why keep functions short? Same reason.

Rust reduces the amount of state I need to keep track of in my brain. It checks things that before I depended on myself to check. It gives me tools to express what I want in fewer lines, but still allows maximum control when needed. The same functionality with less code, and checks to ensure it’s better code, these make me more productive and introduce fewer bugs.

Strong types help the compiler help you

Strong typing gives the compiler information it can use to spot errors. This is important as a program grows from a toy into a useful thing. Assumptions within the code change and strong typing check the assumptions so that each version of the program globally either uses the old, or the new assumptions, but not both.

The key to this is being able to describe to the compiler the intended constraints of our code as clearly as possible.

Expressive types prevent needing type “escape hatches”

One problem with weakly-typed languages is when you need to do something that the type system doesn’t quite let you describe. This leads to needing to use the language’s escape hatches, the “do what I mean!” outs, like casting, that let you do what you need to do, but also inherently create places where the compiler can’t help check things.

Or, there may be ambiguity because a type serves two purposes, depending on the context. One example would be returning a pointer. Is NULL a “good” return value, or is it an error? The programmer needs to know based upon external information (docs), and the compiler can’t help by checking the return value is used properly. Or, say a function returns int. Usually a negative value is an error, but not always. And, negative values are nonzero so they evaluate as true in a conditional statement! It’s just…loose.

Rust distinguishes between valid and error results much more explicitly. It has a richer type system with sum types like Option and Result. These eliminate using a single value for both error and success return cases, and let the compiler help the programmer get it right. A richer type system lets us avoid needing escapes.

Memory Safety, Lifetimes, and the Borrow Checker

For me, this is another case where Rust is enabling the verification of something that C programmers learned painfully how to do right — or else. In C I’ve had functions that “borrowed” a pointer versus ones that “took ownership”, but this was not enforced in the language, only documented in the occasional comment above the function that it had one behavior or the other. So for me it was like “duh”, yeah, we notate this, have terms that express what’s happening, and the compiler can check it. Having to use ref-counting or garbage collection is great but for most cases it’s not strictly needed. And if we do need it, it’s available.

Cargo and Libraries

Cargo makes using libraries easy. Easy libraries mean your program can focus more on doing its thing, and in turn make it easier for others to use what you provide. Efficient use of libraries reduce duplicated work and keep lines of code down.

Functional Code, Functional thinking

I like iterators and methods like map, filter, zip, and chain because they make it easier to break down what I’m doing to a sequence into easier to understand fundamental steps, and also make it easier for other coders to understand the code’s intent.

Rewrite everything?

It’s starting to be a cliche. Let’s rewrite everything in Rust! OpenSSL, Tor, the kernel, the browser. Wheeee! Of course this isn’t realistic, but why do people exposed to Rust keep thinking things would be better off in Rust?

I think for two reasons, each from a different group. First, from coders coming from Python, Ruby, and JavaScript, Rust offers some of the higher-level conveniences and productivity they expect. They’re familiar with Rust development model and its Cargo-based, GitHub-powered ecosystem. Types and the borrow checker are a learning curve for them, but the result is blazingly fast code, and the ability to do systems-level things, like calling ioctls, in which a C extension would’ve been called for — but these people don’t want to learn C. These people might call for a rewrite in Rust because it brings that component into the realm of things they can hack on.

Second, there are people like me, people working in C and Python on Linux systems-level stuff — the “plumbing”, who are frustrated with low productivity. C and Python have diametrically-opposed advantages and disadvantages. C is fast to run but slow to write, and hard to write securely. Python is more productive but too slow and RAM-hungry for something running all the time, on every system. We must deal with getting C components to talk to Python components all the time, and it isn’t fun. Rust is the first language that gives a system programmer performance and productivity. These people might see Rust as a chance to increase security, to increase their own productivity, to never have to touch libtool/autoconf ever again, and to solve the C/Python dilemma with a one language solution.

Incremental Evolution of the Linux Platform

Fun to think about for a coder, but then, what, now we have C, Python, AND Rust that need to interact? “If only everything were Rust… and it would be so easy… how hard could it be?” 🙂 Even in Rust, a huge, not-terribly-fun task. I think Rust has great promise, but success lies in incremental evolution of the Linux platform. We’ve seen service consolidation in systemd and the idea of Linux as a platform distinct from Unix. We have a very useful language-agnostic IPC mechanism — DBus — that gives us more freedom to link things written in new languages. I’m hopeful Rust can find places it can be useful as Linux gains new capabilities, and then perhaps converting existing components may happen as maintainers gain exposure and experience with Rust, and recognize its virtues.

The Future

Rust is not standing still. Recent developments like native debugging support in GDB, and the ongoing MIR work, show that Rust will become even better over time. But don’t wait. Rust can be used to rapidly develop high-quality programs today. Learning Rust can benefit you now, and also yield dividends as Rust and its ecosystem continue to improve.