Block devices and Linux

1. Block devices and Linux • Linux has a generic block device layer with which all filesystems will interact. • SCSI is no different in this regard – it registers itself with the block device layer so it can receive requests. • SCSI also handles character device requests and ioctls that do not originate in the block device layer.

2. What is the “Mid-Layer”? • Linux SCSI support can be viewed as 3 levels. • Upper level is device management, such as tape, cdrom, disk, etc. • Lower level talks to host adapters. • Middle layer is essentially a traffic cop, handing requests from rest of kernel, and dispatching them to the rest of SCSI.

3. What was wrong in 2.2? • The elevator algorithms in 2.2 allowed requests to grow irregardless of the capabilities of the underlying device. • All SCSI disks were handled in a single queue. • Disk driver had to split requests that had become too large. • One set of common logic for verifying requests had not become too large.

4. What was wrong in 2.2 (cont) • Character device requests not in queue. • SMP safety was clumsily handled, leading to race conditions and poor performance. • Poor scalability. • Many drivers continue to use old error handling code.

5. Changes for Linux-2.4 • Block device layer was generalized to support a “request_queue_t” abstract datatype that represents a queue. • Contains function pointers that drivers can use for managing the size of requests inserted into queues. • Requests no longer can grow to be too large to be handled at one time.

6. Changes for 2.4 (cont) • No longer any need for splitting requests. • No need for ugly logic to scan a queue for a queueable request. • SMP locking in mid-layer cleaned up to provide finer granularity.

7. Changes for 2.4 (cont) • A SCSI queuing library was created – a set of functions for queue management that are tailored to different sets of requirements. • SCSI was modified to use a single queue for each physical device. • Character device requests and ioctls are inserted into the same queue at the tail, and handled the same as other requests.

8. Upcoming changes for 2.5 • All drivers will be forced to use new error handling code. • Disk driver will be updated to handle larger number of disks. • SMP locking will be cleaned up some more to improve scalability.

9. SMP Cleanups (cont). • Low-level drivers don’t need to protect queue – they don’t have access to it. • Each low-level driver should have a separate lock – ideally one per instance of host, but could be a driver-wide lock initially. This should be up to the low-level driver.

10. Large numbers of disks • Current disk driver allocates 8 majors, allowing for only 128 disks. • Plans are in the works to allow disk driver to dynamically allocate major numbers. • Would support up to about 4000 disks, when major numbers are exhausted. • Possible to go beyond this by using fewer bits for partitions.

11. Wish list. • Implement some SCSI-3 features (larger commands, sense buffers). • Improve support for shared busses. • Support target-mode. • Check module add/remove code for SMP safety, implement locks. • Improvements related to high-availability.

12. Conclusions The major goal of a rewrite of SCSI queuing has been accomplished. A number of architectural problems were resolved at the same time. There are still some interesting tasks still to be addressed for 2.5. See http://www.andante.org/scsi.html for more info, and http://www.andante.org/scsi_todo.html for “todo” list.