As mentioned in the beta release notes, the kernel in RHEL 7.2 contains a rebased LIO kernel target, to the equivalent of the Linux 4.0.stable series.
This is a big update. LIO has improved greatly since 3.10. It has added support for SCSI features that enable VMWare VAAI support, as well as data integrity (DIF), and significant iSER work, for those of you using Infiniband. (SRP is also supported, as well as iSCSI and FCoE, of course.)
Note that we still do not ship support for the Fibre Channel qla2xxx fabric. It still seems to be something storage vendors and integrators want, more than a feature our customers are telling us they want in RHEL.
(On a side note, Infiniband hardware is pretty affordable these days! For all you datacenter hobbyists who have a rack in the garage, I might suggest a cheap previous-gen IB setup and either SRP or iSER as the way to go and still get really high IOPs.)
Users of RHEL 7’s SCSI target should find RHEL 7.2 to be a very nice upgrade. Please try the beta out and report any issues you find of course, but it’s looking really good so far.
When doing kernel development, doing it in a virtual machine can be very convenient, if there’s no need for actual hardware devices or features. This is especially true for network or client/server development where multiple physical machines would otherwise be needed. Plus, VMs reboot much faster than actual hardware!
The #1 tip: a shared development directory
My preferred setup is to use KVM via virt-manager. I use my editor and the compiler on the host, and then mount my development directory on the guest, and then install the compiled modules there. This lets development on the host remain undisturbed by unstable kernel versions and new target distro versions. In fact, my host is still on RHEL 6, although I’m working on features for much more current kernels and distro releases.
I use NFS to export a mount point, and then mount it in an identical location in each guest. Then, edit and build the kernel on the host, using the ‘O’ kernel make option to keep .config and build files separate from the kernel’s git tree, although both the source and build dirs are under the mount point accessible to the guest. Finally on the guest, “
make O=/path/to/buildfiles modules_install install” and everything’s ready to test.
On a more recent host, and with guests that support it, an easier way to set up a shared directory would be using VirtFS. NFS is a little fiddly to set up for the first time, and virtfs looks pretty easy, and a little faster and secure, even.
- Guest debug output onto the host. Set up a virtual serial port and point it at a file on the host. Then, add “
console=ttyS0,115200 console=tty0” to the guest kernel command line. This will output everything to the file as well as keep outputting to the guest console. Then ‘
tail -f’ the file, and you can be assured any kernel oopses or other messages will be captured. The file will be truncated every time the guest is restarted, BTW.
- Turn on all relevant debugging options (under “kernel hacking”) when compiling your kernel. If you start with the distribution’s .config file, many won’t be set. I’d recommend turning everything on. Also turn on frame pointers, and configure out drivers and subsystems the guest won’t need. ‘
make localmodconfig’ (run from the guest) might help here.
- If you are in an edit/compile loop, use the ‘M=’ make option to just build where you’re hacking, and save make from scanning the whole tree. Just use ‘
make modules’ (on the host) and ‘
make modules_install’ (on the guest) to save time (posttest??? arrgh) and not reinstall a kernel if it hasn’t actually changed. The build will increment the kernel version, which is printed during build and in ‘
uname -v’ output, only if it actually built a new kernel image.
- Get an SSD. Pays for itself in saved time almost instantly.
- Use ccache. If a source file has already been built with identical headers, ccache keeps the object around and saves the build from doing repeated work. This is as simple as installing ccache on the host, and then setting ‘
export CC="ccache gcc"’ in your .bash_profile.
- Make sure your guest has two or more virtual CPUs, in order to properly expose yourself to races when testing.
- Set guests to auto-login and turn off screen savers. Use Ctrl-R (reverse history search) in bash aggressively when repeating test steps on the guest. Give different guests different-colored backgrounds to tell them apart more easily.
- Make sure lockdep checking is enabled, and hasn’t fired earlier in the boot process, because then it turns itself off.
- Familiarize yourself with magic-sysrq feature in Documentation/sysrq.txt, and poking it via /proc/sysrq-trigger.
I hope some of this is helpful, and I’d love to know about more tips and techniques people have.
EDIT: Some more tips in G+ comments here!
RHEL 7 will be using the LIO kernel target subsystem for iSCSI and other protocols, instead of the tgtd daemon, aka scsi-target-utils. This is a change from RHEL 6, where we use tgtd for iSCSI target support, and LIO only for FCoE targets, via the fcoe-target-utils package.
Users of tgtd can prepare for RHEL 7 by trying Fedora 17 or 18, which have current LIO/targetcli code. LIO has many features tgtd doesn’t, but LIO also won’t cover 100% of tgtd’s features at first, either. File bugs for these regressions and we’ll work to address them.
Other software that currently uses tgtd may wish to look at the rtslib Python library, and targetcli’s JSON config format — LIO has a nice API so if you find yourself wanting to parse targetcli output, please hold off and email me instead, ok? 🙂
scsi-target-utils will still be available via EPEL for RHEL 7, and supported in RHEL 6 for its lifetime.