Poster of Linux kernelThe best gift for a Linux geek
 Linux kernel map 
⇦ prev ⇱ home next ⇨

14.7. Hotplug

There are two different ways to view hotplugging. The kernel views hotplugging as an interaction between the hardware, the kernel, and the kernel driver. Users view hotplugging as the interaction between the kernel and user space through the program called /sbin/hotplug. This program is called by the kernel when it wants to notify user space that some type of hotplug event has just happened within the kernel.

14.7.1. Dynamic Devices

The most commonly used meaning of the term "hotplug" happens when discussing the fact that most all computer systems can now handle devices appearing or disappearing while the system is powered on. This is very different from the computer systems of only a few years ago, where the programmers knew that they needed to scan for all devices only at boot time, and they never had to worry about their devices disappearing until the power was turned off to the whole machine. Now, with the advent of USB, CardBus, PCMCIA, IEEE1394, and PCI Hotplug controllers, the Linux kernel needs to be able to reliably run no matter what hardware is added or removed from the system. This places an added burden on the device driver author, as they must now always handle a device being suddenly ripped out from underneath them without any notice.

Each different bus type handles the loss of a device in a different way. For example, when a PCI, CardBus, or PCMCIA device is removed from the system, it is usually a while before the driver is notified of this action through its remove function. Before that happens, all reads from the PCI bus return all bits set. This means that drivers need to always check the value of the data they read from the PCI bus and properly be able to handle a 0xff value.

An example of this can be seen in the drivers/usb/host/ehci-hcd.c driver, which is a PCI driver for a USB 2.0 (high-speed) controller card. It has the following code in its main handshake loop to detect if the controller card has been removed from the system:

result = readl(ptr);
if (result =  = ~(u32)0)    /* card removed */
    return -ENODEV;

For USB drivers, when the device that a USB driver is bound to is removed from the system, any pending urbs that were submitted to the device start failing with the error -ENODEV. The driver needs to recognize this error and properly clean up any pending I/O if it occurs.

Hotpluggable devices are not limited only to traditional devices such as mice, keyboards, and network cards. There are numerous systems that now support removal and addition of entire CPUs and memory sticks. Fortunately the Linux kernel properly handles the addition and removal of such core "system" devices so that individual device drivers do not need to pay attention to these things.

14.7.2. The /sbin/hotplug Utility

As alluded to earlier in this chapter, whenever a device is added or removed from the system, a "hotplug event" is generated. This means that the kernel calls the user-space program /sbin/hotplug. This program is typically a very small bash script that merely passes execution on to a list of other programs that are placed in the /etc/hotplug.d/ directory tree. For most Linux distributions, this script looks like the following:

DIR="/etc/hotplug.d"
for I in "${DIR}/$1/"*.hotplug "${DIR}/"default/*.hotplug ; do
    if [ -f $I ]; then
        test -x $I && $I $1 ;
    fi
done
exit 1

In other words, the script searches for all programs bearing a .hotplug suffix that might be interested in this event and invokes them, passing to them a number of different environment variables that have been set by the kernel. More details about how the /sbin/hotplug script works can be found in the comments in the program and in the hotplug(8) manpage.

As mentioned previously, /sbin/hotplug is called whenever a kobject is created or destroyed. The hotplug program is called with a single command-line argument providing a name for the event. The core kernel and specific subsystem involved also set a series of environment variables (described below) with information on what has just occurred. These variables are used by the hotplug programs to determine what has just happened in the kernel, and if there is any specific action that should take place.

The command-line argument passed to /sbin/hotplug is the name associated with this hotplug event, as determined by the kset assigned to the kobject. This name can be set by a call to the name function that is part of the kset's hotplug_ops structure described earlier in this chapter; if that function is not present or never called, the name is that of the kset itself.

The default environment variables that are always set for the /sbin/hotplug program are:

ACTION

The string add or remove, depending on whether the object in question was just created or destroyed.

DEVPATH

A directory path, within the sysfs filesystem, that points to the kobject that is being either created or destroyed. Note that the mount point of the sysfs filesystem is not added to this path, so it is up to the user-space program to determine that.

SEQNUM

The sequence number for this hotplug event. The sequence number is a 64-bit number that is incremented for every hotplug event that is generated. This allows user space to sort the hotplug events in the order in which the kernel generates them, as it is possible for a user-space program to be run out of order.

SUBSYSTEM

The same string passed as the command-line argument as described above.

A number of the different bus subsystems all add their own environment variables to the /sbin/hotplug call, when devices associated with the bus are added or removed from the system. They do this in their hotplug callback that is specified in the struct kset_hotplug_ops assigned to their bus (as described in Section 14.3.1). This allows user space to be able to automatically load any necessary module that might be needed to control the device that has been found by the bus. Here is a list of the different bus types and what environment variables they add to the /sbin/hotplug call.

14.7.2.1 IEEE1394 (FireWire)

Any devices on the IEEE1394 bus, also known as Firewire, have the /sbin/hotplug parameter name and the SUBSYSTEM environment variable set to the value ieee1394. The ieee1394 subsystem also always adds the following four environment variables:

VENDOR_ID

The 24-bit vendor ID for the IEEE1394 device

MODEL_ID

The 24-bit model ID for the IEEE1394 device

GUID

The 64-bit GUID for the device

SPECIFIER_ID

The 24-bit value specifying the owner of the protocol spec for this device

VERSION

The value that specifies the version of the protocol spec for this device

14.7.2.2 Networking

All network devices create a hotplug event when the device is registered or unregistered in the kernel. The /sbin/hotplug call has the parameter name and the SUBSYSTEM environment variable set to the value net, and just adds the following environment variable:

INTERFACE

The name of the interface that has been registered or unregistered from the kernel. Examples of this are lo and eth0.

14.7.2.3 PCI

Any devices on the PCI bus have the parameter name and the SUBSYSTEM environment variable set to the value pci. The PCI subsystem also always adds the following four environment variables:

PCI_CLASS

The PCI class number for the device, in hex.

PCI_ID

The PCI vendor and device IDs for the device, in hex, combined in the format vendor:device.

PCI_SUBSYS_ID

The PCI subsystem vendor and subsystem device IDs, combined in the format subsys_vendor:subsys_device.

PCI_SLOT_NAME

The PCI slot "name" that is given to the device by the kernel. It is in the format domain:bus:slot:function. An example might be 0000:00:0d.0.

14.7.2.4 Input

For all input devices (mice, keyboards, joysticks, etc.), a hotplug event is generated when the device is added and removed from the kernel. The /sbin/hotplug parameter and the SUBSYSTEM environment variable are set to the value input. The input subsystem also always adds the following environment variable:

PRODUCT

A multivalue string listing values in hex with no leading zeros. It is in the format bustype:vendor:product:version.

The following environment variables may be present, if the device supports it:

NAME

The name of the input device as given by the device.

PHYS

The device's physical address that the input subsystem gave to this device. It is supposed to be stable, depending on the bus position into which the device was plugged.

EV

KEY

REL

ABS

MSC

LED

SND

FF

These all come from the input device descriptor and are set to the appropriate values if the specific input device supports it.

14.7.2.5 USB

Any devices on the USB bus have the parameter name and the SUBSYSTEM environment variable set to the value usb. The USB subsystem also always adds the following environment variables:

PRODUCT

A string in the format idVendor/idProduct/bcdDevice that specifies those USB device-specific fields

TYPE

A string in the format bDeviceClass/bDeviceSubClass/bDeviceProtocol that specifies those USB device-specific fields

If the bDeviceClass field is set to 0, the following environment variable is also set:

INTERFACE

A string in the format bInterfaceClass/bInterfaceSubClass/bInterfaceProtocol that specifies those USB device-specific fields.

If the kernel build option, CONFIG_USB_DEVICEFS, which selects the usbfs filesystem to be built in the kernel, is selected, the following environment variable is also set:

DEVICE

A string that shows where in the usbfs filesystem the device is located. This string is in the format /proc/bus/usb/USB_BUS_NUMBER/USB_DEVICE_NUMBER, in which USB_BUS_NUMBER is the three-digit number of the USB bus that the device is on, and USB_DEVICE_NUMBER is the three-digit number that has been assigned by the kernel to that USB device.

14.7.2.6 SCSI

All SCSI devices create a hotplug event when the SCSI device is created or removed from the kernel. The /sbin/hotplug call has the parameter name and the SUBSYSTEM environment variable set to the value scsi for every SCSI device that is added or removed from the system. There are no additional environment variables added by the SCSI system, but it is mentioned here because there is a SCSI-specific user-space script that can determine what SCSI drivers (disk, tape, generic, etc.) should be loaded for the specified SCSI device.

14.7.2.7 Laptop docking stations

If a Plug-and-Play-supported laptop docking station is added or removed from the running Linux system (by inserting the laptop into the station, or removing it), a hotplug event is created. The /sbin/hotplug call has the parameter name and the SUBSYSTEM environment variable set to the value dock. No other environment variables are set.

14.7.2.8 S/390 and zSeries

On the S/390 architecture, the channel bus architecture supports a wide range of hardware, all of which generate /sbin/hotplug events when they are added or removed from the Linux virtual system. These devices all have the /sbin/hotplug parameter name and the SUBSYSTEM environment variable set to the value dasd. No other environment variables are set.

14.7.3. Using /sbin/hotplug

Now that the Linux kernel is calling /sbin/hotplug for every device added and removed from the kernel, a number of very useful tools have been created in user space that take advantage of this. Two of the most popular tools are the Linux Hotplug scripts and udev.

14.7.3.1 Linux hotplug scripts

The Linux hotplug scripts started out as the very first user of the /sbin/hotplug call. These scripts look at the different environment variables that the kernel sets to describe the device that was just discovered and then tries to find a kernel module that matches up with that device.

As has been described before, when a driver uses the MODULE_DEVICE_TABLE macro, the program, depmod, takes that information and creates the files located in /lib/module/KERNEL_VERSION/modules.*map. The * is different, depending on the bus type that the driver supports. Currently, the module map files are generated for drivers that work for devices that support the PCI, USB, IEEE1394, INPUT, ISAPNP, and CCW subsystems.

The hotplug scripts use these module map text files to determine what module to try to load to support the device that was recently discovered by the kernel. They load all modules and do not stop at the first match, in order to let the kernel work out what module works best. These scripts do not unload any modules when devices are removed. If they were to try to do that, they could accidentally shut down devices that were also controlled by the same driver of the device that was removed.

Note, now that the modprobe program can read the MODULE_DEVICE_TABLE information directly from the modules without the need of the module map files, the hotplug scripts might be reduced to a small wrapper around the modprobe program.

14.7.3.2 udev

One of the main reasons for creating the unified driver model in the kernel was to allow user space to manage the /dev tree in a dynamic fashion. This had previously been done in user space with the implementation of devfs, but that code base has slowly rotted away, due to a lack of an active maintainer and some unfixable core bugs. A number of kernel developers realized that if all device information was exported to user space, it could perform all the necessary management of the /dev tree.

devfs has some very fundamental flaws in its design. It requires every device driver to be modified to support it, and it requires that device driver to specify the name and location within the /dev tree where it is placed. It also does not properly handle dynamic major and minor numbers, and it does not allow user space to override the naming of a device in a simple manner, forcing the device naming policy to reside within the kernel and not in user space. Linux kernel developers really hate having policy within the kernel, and since the devfs naming policy does not follow the Linux Standard Base specification, it really bothers them.

As the Linux kernel started to be installed on huge servers, a lot of users ran into the problem of how to manage very large numbers of devices. Disk drive arrays of over 10,000 unique devices presented the very difficult task of ensuring that a specific disk was always named with the same exact name, no matter where it was placed in the disk array or when it was discovered by the kernel. This same problem also plagued desktop users who tried to plug two USB printers into their system and then realized that they had no way of ensuring that the printer known as /dev/lpt0 would not change and be assigned to the other printer if the system was rebooted.

So, udev was created. It relies on all device information being exported to user space through sysfs and on being notified by /sbin/hotplug that a device was added or removed. Policy decisions, such as what name to give a device, can be specified in user space, outside of the kernel. This ensures that the naming policy is removed from the kernel and allows a large amount of flexibility about the name of each device.

For more information on how to use udev and how to configure it, please see the documentation that comes included with the udev package in your distribution.

All that a device driver needs to do, for udev to work properly with it, is ensure that any major and minor numbers assigned to a device controlled by the driver are exported to user space through sysfs. For any driver that uses a subsystem to assign it a major and minor number, this is already done by the subsystem, and the driver doesn't have to do any work. Examples of subsystems that do this are the tty, misc, usb, input, scsi, block, i2c, network, and frame buffer subsystems. If your driver handles getting a major and minor number on its own, through a call to the cdev_init function or the older register_chrdev function, the driver needs to be modified in order for udev to work properly with it.

udev looks for a file called dev in the /class/ tree of sysfs, in order to determine what major and minor number is assigned to a specific device when it is called by the kernel through the /sbin/hotplug interface. A device driver merely needs to create that file for every device it controls. The class_simple interface is usually the easiest way to do this.

As mentioned in Section 14.5.1 the first step in using the class_simple interface is to create a struct class_simple with a call to the class_simple_create function:

static struct class_simple *foo_class;
...
foo_class = class_simple_create(THIS_MODULE, "foo");
if (IS_ERR(foo_class)) {
    printk(KERN_ERR "Error creating foo class.\n");
    goto error;
}

This code creates a directory in sysfs in /sys/class/foo.

Whenever a new device is found by your driver, and you assign it a minor number as described in Chapter 3, the driver should call the class_simple_device_add function:

class_simple_device_add(foo_class, MKDEV(FOO_MAJOR, minor), NULL, "foo%d", minor);

This code causes a subdirectory under /sys/class/foo to be created called fooN, where N is the minor number for this device. There is one file created in this directory, dev, which is exactly what udev needs in order to create a device node for your device.

When your driver is unbound from a device, and you give up the minor number that it was attached to, a call to class_simple_device_remove is needed to remove the sysfs entries for this device:

class_simple_device_remove(MKDEV(FOO_MAJOR, minor));

Later, when your entire driver is being shut down, a call to class_simple_destroy is needed to remove the class that you created originally with the call to class_simple_create:

class_simple_destroy(foo_class);

The dev file that is created by the call to class_simple_device_add consists of the major and minor numbers, separated by a : character. If your driver does not want to use the class_simple interface because you want to provide other files within the class directory for the subsystem, use the print_dev_t function to properly format the major and minor number for the specific device.

    ⇦ prev ⇱ home next ⇨
    Poster of Linux kernelThe best gift for a Linux geek