Tag Archives: development

Some targetcli and TCMU questions

Just got an email full of interesting questions, I hope the author will be ok with me answering them here so future searches will see them:

I searched on internet and I don’t find some relevant info about gluster api support via tcmu-runner. Can you tell me please if this support will be added to the stable redhat targetcli in the near future? And I want to know also which targetcli is recommended for setup (targetcli or targetcli-fb) and what is the status for targetcli-3.0.

tcmu-runner is a userspace daemon add-on to LIO that allows requests for a device to be handled by a user process. tcmu-runner has early support for using glfs (via gfapi). Both tcmu-runner and its glfs plugin are beta-quality and will need further work before they are ready for stable Fedora, much less a RHEL release. tcmu-runner just landed in Rawhide, but this is really just to make it easier to test.

RHEL & Fedora use targetcli-fb, which is a fork of targetcli, and what I work on. Since I’m working on both tcmu-runner and targetcli-fb, targetcli-fb will see TCMU support very early.

The -fb packages I maintain switched to a “fbXX” version scheme, so I think you must be referring to the other one 🙂 I don’t have any info about the RTS/Datera targetcli’s status, other than nobody likes having two versions, the targetcli maintainer and I have discussed unifying them into a common version, but the un-fun work of merging them has not happened yet.

Emacs and using multiple C code styles

I primarily work on Linux, so I put this in my Emacs config:

; Linux mode for C
(setq c-default-style
      '((c-mode . "linux") (other . "gnu")))

However, other projects like QEMU have their own style preferences. So here’s what I added to use a different style for that. First, I found the qemu C style defined here. Then, to only use this on some C code, we attach a hook that only overrides the default C style if the filename contains “qemu”, an imperfect but decent-enough test.

(defconst qemu-c-style
  '((indent-tabs-mode . nil)
    (c-basic-offset . 4)
    (tab-width . 8)
    (c-comment-only-line-offset . 0)
    (c-hanging-braces-alist . ((substatement-open before after)))
    (c-offsets-alist . ((statement-block-intro . +)
                        (substatement-open . 0)
                        (label . 0)
                        (statement-cont . +)
                        (innamespace . 0)
                        (inline-open . 0)
    (c-hanging-braces-alist .
                             (block-close . c-snug-do-while)
                             ;; structs have hanging braces on open
                             (class-open . (after))
                             ;; ditto if statements
                             (substatement-open . (after))
                             ;; and no auto newline at the end
  "QEMU C Programming Style")

(c-add-style "qemu" qemu-c-style)

(defun maybe-qemu-style ()
  (when (and buffer-file-name
       (string-match "qemu" buffer-file-name))
    (c-set-style "qemu")))

(add-hook 'c-mode-hook 'maybe-qemu-style)

Kernel Development with VMs: Some Tips

When doing kernel development, doing it in a virtual machine can be very convenient, if there’s no need for actual hardware devices or features. This is especially true for network or client/server development where multiple physical machines would otherwise be needed. Plus, VMs reboot much faster than actual hardware!

The #1 tip: a shared development directory

My preferred setup is to use KVM via virt-manager. I use my editor and the compiler on the host, and then mount my development directory on the guest, and then install the compiled modules there. This lets development on the host remain undisturbed by unstable kernel versions and new target distro versions. In fact, my host is still on RHEL 6, although I’m working on features for much more current kernels and distro releases.

I use NFS to export a mount point, and then mount it in an identical location in each guest. Then, edit and build the kernel on the host, using the ‘O’ kernel make option to keep .config and build files separate from the kernel’s git tree, although both the source and build dirs are under the mount point accessible to the guest. Finally on the guest, “make O=/path/to/buildfiles modules_install install” and everything’s ready to test.

On a more recent host, and with guests that support it, an easier way to set up a shared directory would be using VirtFS. NFS is a little fiddly to set up for the first time, and virtfs looks pretty easy, and a little faster and secure, even.

Other tips

  1. Guest debug output onto the host. Set up a virtual serial port and point it at a file on the host. Then, add “console=ttyS0,115200 console=tty0” to the guest kernel command line. This will output everything to the file as well as keep outputting to the guest console. Then ‘tail -f’ the file, and you can be assured any kernel oopses or other messages will be captured. The file will be truncated every time the guest is restarted, BTW.
  2. Turn on all relevant debugging options (under “kernel hacking”) when compiling your kernel. If you start with the distribution’s .config file, many won’t be set. I’d recommend turning everything on. Also turn on frame pointers, and configure out drivers and subsystems the guest won’t need. ‘make localmodconfig’ (run from the guest) might help here.
  3. If you are in an edit/compile loop, use the ‘M=’ make option to just build where you’re hacking, and save make from scanning the whole tree. Just use ‘make modules’ (on the host) and ‘make modules_install’ (on the guest) to save time (posttest??? arrgh) and not reinstall a kernel if it hasn’t actually changed. The build will increment the kernel version, which is printed during build and in ‘uname -v’ output, only if it actually built a new kernel image.
  4. Get an SSD. Pays for itself in saved time almost instantly.
  5. Use ccache. If a source file has already been built with identical headers, ccache keeps the object around and saves the build from doing repeated work. This is as simple as installing ccache on the host, and then setting ‘export CC="ccache gcc"’ in your .bash_profile.
  6. Make sure your guest has two or more virtual CPUs, in order to properly expose yourself to races when testing.
  7. Set guests to auto-login and turn off screen savers. Use Ctrl-R (reverse history search) in bash aggressively when repeating test steps on the guest. Give different guests different-colored backgrounds to tell them apart more easily.
  8. Make sure lockdep checking is enabled, and hasn’t fired earlier in the boot process, because then it turns itself off.
  9. Familiarize yourself with magic-sysrq feature in Documentation/sysrq.txt, and poking it via /proc/sysrq-trigger.

I hope some of this is helpful, and I’d love to know about more tips and techniques people have.

EDIT: Some more tips in G+ comments here!