CS6900 Independent study

1. CS6900 Independent study Bolin Hsu <bolin_hsu@yahoo.com>

2. Introduction • Relationship of Linux kernel and Linux device drivers • Implementation of a Linux device driver • How to write your own device driver • Bibliography Bolin Hsu <bolin_hsu@yahoo.com>

3. Concepts

4. Objective • Let the users access the device by file system calls: … fd=open(“/dev/myrd”,O_RDWR); … write(fd,buffer,sizeof(buffer)); … read(fd,buffer,N); … close(fd); Bolin Hsu <bolin_hsu@yahoo.com>

5. Challenges • How to connect the device file “/dev/myrd” to the device • How to make the system calls access the device Bolin Hsu <bolin_hsu@yahoo.com>

6. Linking of device file and the device Bolin Hsu <bolin_hsu@yahoo.com>

7. Everything in UNIX is a file • User interact with the device through filesystem functions • open, close, read, write, ioctl, etc • A device file or a filesystem • Two interfaces to the device • Interface to user: device file • Interface to kernel: service routine Bolin Hsu <bolin_hsu@yahoo.com>

8. Device file concept • Device file type: • Character (inode.type = 3) • Block (inode.type = 4) • Device file number • Major: device type (2: floppy disk 3: hard disk, etc) • Minor: device number • Examples: brw-rw---- 1 root disk 3, 0 May 19 2004 /dev/had Bolin Hsu <bolin_hsu@yahoo.com>

9. Creating the device file • Create our ram disk device file • Type is block • Major number is 100 • Minor number is 0 % mknod /dev/ b 100 0 • Another option is to create device during device driver initialization. Bolin Hsu <bolin_hsu@yahoo.com>

10. Connecting system calls to the device driver Bolin Hsu <bolin_hsu@yahoo.com>

11. How Linux kernel uses device drivers User applications Libraries User mode Kernel mode File subsystem Buffer/page cache Character device driver Block device driver Hardware control Bolin Hsu <bolin_hsu@yahoo.com>

12. Block I/O and Page I/O • Two fundamental I/O type: • Block I/O: bread() • Page I/O: brw_page() • Block I/O: read/write a single block. Kernel uses it to handle file system superblock and inode. • Page I/O: read/write a page in a file. Used in read/write system calls. Bolin Hsu <bolin_hsu@yahoo.com>

13. Page/buffer cache • A cache is a RAM area • The cache holds data which usually should be on a block device. • I/O from the cache is faster than the device. • Kernel can merge and schedule device I/O to improve system performance. Bolin Hsu <bolin_hsu@yahoo.com>

14. Sector, block, and buffer cache • A sector is the basic data transfer unit of a block device • A block is a collection of adjacent sectors transferred in one I/O request • Every block device file has its own block size • Every block has a corresponding buffer cache Bolin Hsu <bolin_hsu@yahoo.com>

15. Buffer cache in RAM Blocks on device How device request uses buffer cache request Buffer head Buffer head Buffer head request Bolin Hsu <bolin_hsu@yahoo.com>

16. Device driver API calling sequence open close read write device driver API block device switch table buffer cache calls open release request ioctl check_media_change Interrupt service ruotine revalidate Interrupt vector device interrupt Bolin Hsu <bolin_hsu@yahoo.com>

17. How read() works • The read() system call searches the page in the page cache. If the page is in the cache and valid, it is copied to the user buffer. If no valid page is found in the cache, the process issues a request for the page and suspend itself. • When the page becomes available later, the kernel wakes up the suspended process. The process then copies the page to the user buffer. Bolin Hsu <bolin_hsu@yahoo.com>

18. Trace read() system call • These functions are called in order: • read() • sys_read() • def_blk_fops.read: generic_file_read() • do_generic_file_read()if the page is found in the page cache, it is copied to the user buffer. If it is not in the cache, call next function. This function could suspend the calling process. • def_blk_aops.readpage: blkdev_readpage() • block_read_full_page() • blkdev_get_block() Bolin Hsu <bolin_hsu@yahoo.com>

19. How write() works • The write() system call copies the user buffer to the page cache and returns. • Unlike read(), the write() system call doesn’t wait for the transfer between page cache and the device to complete. There is no data dependency in write operation. Bolin Hsu <bolin_hsu@yahoo.com>

20. Trace write() system call • These functions are called in order: • write() • sys_write() • def_blk_fops.write: generic_file_write() • def_blk_aops.prepare_write: blkdev_prepare_write()def_blk_aops.commit_write: blkdev_commit_write() • block_prepare_write()block_commit_write() Bolin Hsu <bolin_hsu@yahoo.com>

21. How open() works • open() makes the file subsystem read the inode of the file from the device. • The kernel then checks the inode type and knows it is a device file. The kernel then initializes the inode structure in RAM to contain file operations defined in the device driver. The kernel uses the major number of the device to find the file operations. • From this point on, file operation calls to this inode will cause the execution of the corresponding device driver functions. Bolin Hsu <bolin_hsu@yahoo.com>

22. Trace open() system call • These functions are called in order: • open() • sys_open() • file_open() • dengry_open() • def_blk_fops.open: blkdev_open() • do_open() • get_blkfops()retrieves blkdevs[MAJOR].bdops, stores in bdev.bd_op Bolin Hsu <bolin_hsu@yahoo.com>

23. The request queue of the device • Requests to a block device is maintained in a request queue of the device. • BLK_DEFAULT_QUEUE(MAJOR) finds the request queue by the major number. • A request function from the device driver handles the request. • The request function is registered when the queue is initialized:blk_init_queue(BLK_DEFAULT_QUEUE(MAJOR),request_function); Bolin Hsu <bolin_hsu@yahoo.com>

24. Implementation

25. What’s in a device driver • A device driver is like a server. It has: • A set of functions to services • Functions to publish the services to the kernel • Some internal data structures for bookkeeping purposes Bolin Hsu <bolin_hsu@yahoo.com>

26. In-kernel or kernel module • Two ways of adding device driver to the kernel: • Compile into the kernel • Loadable kernel module • Benefit of kernel module • No need to rebuild kernel • No need to reboot system Bolin Hsu <bolin_hsu@yahoo.com>

27. Block device interface to Linux kernel /usr/src/linux/fs/block_dev.c struct file_operations def_blk_fops = { open: blkdev_open, release: blkdev_close, llseek: block_llseek, read: generic_file_read, write: generic_file_write, mmap: generic_file_mmap, fsync: block_fsync, ioctl: blkdev_ioctl, }; // Warning: nonstandard gcc structure initialization Bolin Hsu <bolin_hsu@yahoo.com>

28. File system interface of the block device /usr/src/linux/include/linux/fs.h struct block_device_operations { int (*open) (struct inode *, struct file *); int (*release) (struct inode *, struct file *); int (*ioctl) (struct inode *, struct file *, unsigned, unsigned long); int (*check_media_change) (kdev_t); int (*revalidate) (kdev_t); struct module *owner; };// file system hooks for the block devices Bolin Hsu <bolin_hsu@yahoo.com>

29. Important global array • read_ahead // how many blockblk_size // device sizeblksize_size // block sizehardsect_size // sector size • These arrays are indexed by major number • Kernel fetches information about the device from these arrays Bolin Hsu <bolin_hsu@yahoo.com>

30. Initializing the device driver • Done in init_module • Allocate memory • Initialize the device driver’s slots in global arrays • Initialize the request queue of the device driver • Register the device driver (API) Bolin Hsu <bolin_hsu@yahoo.com>

31. Initialization example Int Init_module(void) { ... myrd_storage = (char *)kmalloc(2*1024*1024*sizeof(char), GFP_KERNEL); ... read_ahead[MAJOR_NR] = 2; // sectors blk_size[MAJOR_NR] = kmalloc(sizeof(int), GFP_KERNEL); *blk_size[MAJOR_NR] = 2048; // device size (kilobytes); blksize_size[MAJOR_NR] = kmalloc(sizeof(int), GFP_KERNEL); *blksize_size[MAJOR_NR] = 512; // block size (bytes) hardsect_size[MAJOR_NR] = kmalloc(sizeof(int), GFP_KERNEL); *hardsect_size[MAJOR_NR] = 512; // sector size (bytes) blk_init_queue(BLK_DEFAULT_QUEUE(MAJOR_NR), myrd_request); register_blkdev(MAJOR_NR,"myrd",&myrd_ops); register_disk(NULL, MKDEV(MAJOR_NR, 0), 1, &myrd_ops, 100); ... } Bolin Hsu <bolin_hsu@yahoo.com>

32. Cleanup example void cleanup_module(void) { ... fsync_dev(MKDEV(MAJOR_NR, 0)); blk_cleanup_queue(BLK_DEFAULT_QUEUE(MAJOR_NR)); kfree(hardsect_size[MAJOR_NR]); hardsect_size[MAJOR_NR] = NULL; kfree(blk_size[MAJOR_NR]); blk_size[MAJOR_NR] = NULL; kfree(blksize_size[MAJOR_NR]); blksize_size[MAJOR_NR] = NULL; kfree(myrd_storage); unregister_blkdev(MAJOR_NR,"myrd"); } Bolin Hsu <bolin_hsu@yahoo.com>

33. Servicing a request • The kernel passes to the request function the beginning sector and the number of sectors to read. • The request function can calculate the starting location and size by:start_loc = sector * SECTOR_SIZE;size = nr_sectors * SECTOR_SIZE; • Assume the RAM disk stores the data in an byte array data[], the memory region to be copied is data[start_loc] to data[start_loc+size-1] Bolin Hsu <bolin_hsu@yahoo.com>

34. Request routine example void myrd_request(request_queue_t *q) { ... while(1) { INIT_REQUEST; // this macro does a return when queue is empty location = myrd_storage + CURRENT->sector * MYRD_HARDSECT_SIZE; size = CURRENT->current_nr_sectors * MYRD_HARDSECT_SIZE; spin_lock(&myrd_lock); if (CURRENT->cmd == READ) memcpy(CURRENT->buffer, location, size); else if (CURRENT->cmd == WRITE) memcpy(location, CURRENT->buffer, size); spin_unlock(&myrd_lock); end_request(1); // update the queue and requests } } Bolin Hsu <bolin_hsu@yahoo.com>

35. Adding the device driver into the Linux source tree • Modify the Config.in in the directory • Let user choose Y/N/M for this question:tristate “Configure My RAM disk’ CONFIG_MYRD • Add your code to the makefile • obj_$(CONFIG_MYRD) += myrd.o • The above line adds the object to obj_m or obj_y, depending on the answer in “make config” Bolin Hsu <bolin_hsu@yahoo.com>

36. Roll your own device driver

37. Helpful courses • If you don’t feel comfortable working with the hardware, these two courses are very helpful • CS3432 Digital circuit design • CS3434 Computer interfacing Bolin Hsu <bolin_hsu@yahoo.com>

38. A checklist of tasks • Create the device file • Define the default file operations: • def_blk_ops • def_chr_fops • Initialize the global data arrays • Register the device driver to the kernel • Cleanup everything when you are done Bolin Hsu <bolin_hsu@yahoo.com>

39. How do I drive a real device • The derive driver interface to the user and the kernel remain the same. • The request function will have to interact with the device. This is usually done by reading and writing to device registers. • If the device is slow, you may want to use the interrupt driven approach. The process sets up an I/O operation and suspends. When the I/O completes, the device raise an interrupt request and the interrupt service routine in the device driver is called to handle the completed transfer. Bolin Hsu <bolin_hsu@yahoo.com>

40. How do I drive character device • Needs to supply the file operations defined by the file system. • No cache is used. Thus no need of the request function. • Needs to initialize the global arrays of the kernel and register the driver. • Still needs to implement interrupt service routine. Bolin Hsu <bolin_hsu@yahoo.com>

41. Looking ahead: the 2.6.x kernel • The 2.6 kernel removed several macro used in this presentation. • The 2.6 kernel changed some global bookkeeping arrays. • The 2.6 kernel is preemptive. • The nonstandard structure initialization was removed in 2.6.x Bolin Hsu <bolin_hsu@yahoo.com>

42. Other types of drivers • Sometimes character and block driver models are not suitable • Network interface card driver • Bus driver • PCI, SCSI, etc. • Need to deal with packets like in networking. • And more Bolin Hsu <bolin_hsu@yahoo.com>

43. Suggestions to future work • Build a file system on the ram disk • Partition the data array into boot block, super block, inodes, data blocks. • Supply mount() and unmount() system calls, plus other system calls. • Port the RAM disk driver to kernel 2.6.x Bolin Hsu <bolin_hsu@yahoo.com>

44. Bibliography • Linux device drivers, 2ed AlessandroRubini & Jonathan Corbet, O’Reilly, Sebastopol, CA • Understanding the Linux kernel Daniel P.l Bovet & Marco Cesati, O’Reilly, Sebastopol, CA <ftp://ftp.oreilly.com/pub/examples/linux/drivers2/book_pdf.rar> • The design of the UNIX operating system Maurice J. Bach, Prentice Hall, Englewood Cliffs, New Jersey • Writing a UNIX device driver, 2ed Janet I. Egan, Thomas J. Teixeira, John Wiley & Sons, Inc. Bolin Hsu <bolin_hsu@yahoo.com>