I’ve whipped up a short (7min) screencast on targetd and lsmcli, two new additions to Fedora 18. targetd glues together LVM and LIO to expose a remote API for configuring a system for a storage array role. lsmcli is part of libstoragemgmt, which provides a common way to manage storage arrays from multiple vendors.
If you have Python code that works with kernel modules, please consider using this library in the future. If you have C code that works with modules, you should use libkmod directly! As I’ll be talking about at this years LPC, proper libraries are preferable to calling cmdline progams for low-level stuff, and now there’s one less reason to do so.
Thanks to Jiri Popelka for reviewing the python-kmod package, sorry it took me so long to fix it up 🙂
targetd is up to version 0.2.2 from 0.1, just five weeks later.
- Added manpage
- Added API specification. API support for multiple storage pools.
- Deferred completion for long-running operations
- Volume copy
- Config file format changed to YAML
- Saves configuration across reboots
- Packaged, submitted for inclusion, and accepted for Fedora 18
Second, I’m having trouble implementing SSL support. I searched around and found some pages that talked about easily adding SSL support to Python’s HTTPServer, but after spending a day on it, it still didn’t work. I’m sure this is easy for someone, but that someone is not me 🙂 Anyone care to take a look?
Virtual machines need disk images to run from. While you can put the disk image on the local storage of the machine executing the VM, there are many benefits to centralizing disk images — this lets you seamlessly migrate the VM’s execution between machines, is easier to manage, and can be more space-efficient when many of the disk images are almost the same, which is pretty common.
However, it hasn’t been nearly as easy to provision a new storage volume on the central storage box as it could be. High-end storage appliances have tools for this, but the normal way using only open-source tools has been to ssh into the machine, create the new volume and export it using command–line tools, and then go back to however you’re creating the new VM and tell it about the new volume you created.
targetd is a step towards making this process a little easier.
The remote API is based on jsonrpc-2.0 over HTTP. The Python standard library does much of the work towards implementing our API server, what’s left is the actual implementation of functionality and the jsonrpc error handling.
This API will let targetd eventually tie into existing storage management tools. One such under development is libstoragemgmt, a framework by my colleague Tony Asleson, which will give virtualization tools like oVirt or OpenStack a common API for management of the many proprietary storage appliances, and also open-source appliances like targetd.
Linux has had reliable LVM-based volume management for a long time, and now has an excellent kernel-based storage target subsystem called LIO. targetd uses both of these heavily. In configuring a machine for a storage appliance role, give targetd a volume group to allocate volumes from, set user/password for access, and you’re just about done.
There is one coding pet peeve of mine that I’ve ranted about before that targetd avoids completely. targetd uses libraries to interface with LVM and LIO instead of the all-too-common alternative of passing commands to command-line tools, and parsing the output. Much of the time spent towards targetd was improving these libraries. I believe proper error propagation, reduced text parsing, and better library APIs make this bottom-up approach a long-term win.
Current state and future plans
Today’s announcement is a pre-alpha 0.1 release. I’m really hoping to get to 1.0 for Fedora 18. There is a manpage to write and SSL support to add, but much more important than the source code of targetd is the layout of the remote API itself. I’d love to get some more review of that, as well as the code, before 1.0. I have been working from what I believe are some common use-cases, but more feedback on how admins configure and use storage appliances would be most welcome too!