A hundred other languages want to call your code

The users of a hundred programming languages would like to call your low-level code, but they can’t.

Things have changed in the last 20 years. More people are using languages like Python, Ruby, and a hundred more, that are further from the bare metal. People are building service stacks that tie together many lower-level functions.

Libraries and APIs that make low-level features available to convenient high-level languages (HLLs) are a good thing. As a HL coder, it’s pretty handy to install python-foo, type “import foo” and then have access to that functionality.

What if python-foo isn’t there? HLL users are out of luck, unless they are so determined they make their own python-foo that calls system(), and then parses the output using their language’s fancy text parsing features.

But system() is the devil. We hate system(), folks. If your code calls system() it’s bad, for four reasons:

  1. Overhead. It creates a new process and subshell.
  2. Security. If your code has elevated privileges and is including text input by an untrusted user, watch out. Remember little Bobby Tables, a semicolon is a dangerous thing.
  3. Ease. Parsing command-line programs’ output can be a pain, even if your language helps lessen it. Parsing of errors is even harder and prone to be overlooked.
  4. Portability. A different platform may (or may not) have the program you’re relying on, or its output may be different, and you won’t know.

Early on when I was learning Python, I tried to write a gui for OProfile by parsing its output. OProfile did nice (for the user) things like adding headers on its output, and changing the format of output depending on what it found. Great for users, but it doomed my project. I couldn’t parse the output reliably.

You want to make it easy for the people who are language gurus for each of the hundred languages out there to wrap your functionality without having to become an expert in your code, or even change it. Then the hordes using all the hundred languages can use your library without being an expert in your code or being enough of a guru in their language to write a wrapper. They can just happily use it.

Here’s a positive development, kmod. kmod is a new implementation of the utilities in module-init-tools: modprobe, lsmod, lsmod, etc. Not only does kmod include a libkmod C library, but the commandline programs use it, so we know it works. Yeah! This made it super easy for someone (me) to come along and write a language wrapper (python-kmod) without having to know about module internals. python-kmod makes it easy for Python users to manipulate modules using the friendly language features they’re used to, like exceptions for errors, and lists. If I had been forced to use system(), it probably would have mostly worked, but it would have failed when output parsing failed for some edge case.

I encourage all low-level program writers, my fellow Linux Plumbers, to consider how to make native language bindings possible for your code. You don’t have to write them, just make them possible and you will find all sorts of people calling your code, safely, who couldn’t before.

8 thoughts on “A hundred other languages want to call your code

  1. Mentifex

    The Mentifex artificial intelligence (AiMind) is written simultaneously in low-level Forth and in high-level JavaScript. In order for the “hundred languages” mentioned above to be able to call the AI Mind code, perhaps it will have to be re-written in Perl 🙁

  2. Eli

    Well yeah, but then you have to write a native-code program in a systems language. And That’s Terrible.

  3. agrover Post author

    Eli: We already have all these commandline utilities written in C, but they put the commandline parsing plus cool-thing-they-do in the same binary blob. What I was getting at was if we did a little more work and split them into foo.so, and then /usr/bin/foo that parses the cmdline and invokes the coolstuff in foo.so, then it’s very very easy for other languages to write wrappers cleanly.

    If you have a program written in *one* HLL that might be of use to *other* HLLs, then I’m not sure what you can do. Maybe langs that shared a VM (e.g. Clojure and Groovy) could do cross-lang reuse, but beyond that, no.

    Between Python and Ruby, there’s a lot of idea sharing, but not very much code, I don’t think.

  4. Simon

    It also requires the developer to commit to a stable API/ABI, for what he regards as internal implementation details. And that’s not a decision to be made lightly – it’s not something you do on the chance that someone else *might* want to use your code as a library…

  5. Krishna Carvana

    Definitely believe that which you stated. Your favorite reason appeared to be on the web the simplest thing to be aware of. I say to you, I certainly get irked while people think about worries that they just don’t know about. You managed to hit the nail upon the top as well as defined out the whole thing without having side-effects , people can take a signal. Will likely be back to get more. Thanks

  6. Pingback: Groveronline » Blog Archive » targetd: remote administration of a Linux storage appliance

Leave a Reply

Your email address will not be published. Required fields are marked *