ELF: An Efficient Log-Structured Flash File System For Micro Sensor Nodes

1. ELF: An Efficient Log-Structured Flash File System For Micro Sensor Nodes Hui Dai Michael Neufeld Richard Han University of Colorado at Boulder Computer Science Department ACM SenSys 2004

2. OUTLINE • Introduction • Existing File Systems • Design Issue • ELF File System Architecture • Implementation Issue • Performance Evaluation • Conclusion

3. Building WSN requires the integration of multiple hardware platforms, operating systems, network systems ,and back-end data services It is important to have the option of holding onto data locally until a relaying opportunity arises For some specific application Ex. TinyDB , … Introduction(1/3)

4. Introduction (2/3) • Two challenges in design file system： • Resource constrained • RAM usage • Power consumption • Characteristic of Flash memory • Flash life time

5. Introduction –Goal(3/3) • Allow access to flash memory with simple file operations • Extend the operational lifetime of the flash with wear leveling techniques • Achieve a small memory footprint • Avoid excessive energy consumption • Provide optional best-effort data reliability

6. Existing File Systems

7. Design Issue(1/2) • File Access Behavior • Sensor Data • Form major part of the data stored • Data written sequentially without modifying earlier records • Cleared periodically • Configuration Data • Changes infrequently after deployment • Requires high reliability • Binary Program Images • Dynamic reprogramming • Store the binary image in flash memory before rebooting the system • Requires high reliability

8. Flash Characteristics Flash is divided into sectors (sector = n* 264byte page) On-chip cache (264 bytes) Concurrent read/writes prohibited Reliability mechanisms like logging can either use Flash or EEPROM as is done for ELF’s implementation on Mica2 mote Design Issue(2/2) • Flash Memory Attributes in Mote • EEPROM attributes in atmega128

9. Resource Abstraction Run-time memory In-memory representation of open files Configuration data like cleaning policy General File Operations Logical abstraction of file/dir operation ELF Maintenance Tasks System maintenance tasks like snapshot of dir structure and file meta data in EEPROM ELF File System Architecture Resource Abstraction ELF Maintenance Tasks General File Operations Flash EEPROM RAM

10. Data Structures in RAM, EEPROM and FLASH Node Abstraction File Abstraction File Descriptor RAM Logical View Check point Region Sys Info Node Header EEPROM Page Info Flash Memory Physical Storage

11. Data Structures in RAM, EEPROM and FLASH Node Abstraction • Physical Node • Meta-Data • Unique 16-bit node id • 16-bit version number indicating age • 32-bit field storing length of the node and included data • Types • ELF_DIR – Represents a dir entry in the file system • ELF_FILE – Includes meta-data to operate on file • ELF_COMMON – Represents changes to the file File Abstraction File Descriptor • Per-Page Meta Data struct page_info{ • uint16_t crc; • uint16_t nextPage:11; • uint16_t flags:5; • uint16_t writeEncounter; • uint16_t magicNumber; • } RAM Logical View Check point Region Sys Info Node Header EEPROM Page Info Flash Memory Physical Storage

12. Data Structures in RAM, EEPROM and FLASH Node Abstraction File Abstraction File Descriptor RAM • Node Abstraction • struct elf_node_abstract{ • uint16_t version; • uint32_t beginOffset; • uint32_t locaLen; • uint16_t startPg : 11; • uint16_t flag : 5; • struct elf_node_abstract *next; • }__attribute__((packed)); • File Abstraction • struct elf_file_abstract{ • uint16_t version; • uint16_t inodeNumber; • uint32_t totaLen; • uint8_t flag; • uint8_t writeCounter : 4; • uint8_t readCounter : 4; • uint16_t pInode; • elf_node_abstract_t *first; • }__attribute__((packed)); Logical View Check point Region Sys Info Node Header EEPROM Page Info Flash Memory Physical Storage

13. Data Structures in RAM, EEPROM and FLASH Node Abstraction File Abstraction File Descriptor RAM • File Descriptor • To operate a file • File abstraction • Current offset in that file and the open mode Logical View Check point Region Sys Info • Crash Recovery • Optional • Check point • Maintains snapshots of current operation Node Header EEPROM Page Info Flash Memory Physical Storage

14. File Operations in ELF • Read • Flash->Data • Flash->Cache->Data Open/Create Append Write/Modify • Write • Data->Cache->Flash • Page contents erasedbefore writing • Modify • Page ->Cache->Modify->Flash Read/Seek Rename Delete

15. Resource Management • Bitmap approach to maintain free/dirty/used blocks • Garbage Collection • Cleaner called when number of free pages drop below threshold

16. Performance Evaluation • ELF is implemented on TinyOS • It runs on mica2 Mote

17. Performance Evaluation(1/5) • Sequential Read Performance • First filled the entire memory 500K log file in Flash • Gradually increase the number of bytes read each time • Each run repeated 50 times • MatchBox Read: • FlashCacheRAM • ELF Read： • FlashRAM

18. 16 bytes of sensor data are appended to the file in each write until the whole flash is consumed Experiments performed with varying RAM buffer E1 and E1’ correspond to 0 byte buffer, E2 and E2’ correspond to 32 byte buffer and so on E1 has the whole bitmap stored in RAM whereas E1’ has only first 64 bytes stored in RAM Performance Evaluation(2/5) MatchBox MatchBox

19. Random Read Performance MatchBox doesn’t support random reads For specific application：TinyDB , Dynamic programming Performance Evaluation(3/5)

20. Random Write Performance Write-modify operation Performance Evaluation(4/5)

21. Performance Evaluation(5/5) • Wear Leveling Performance • Primary goal for flash file system • The experiment continues running until the total number of erasures exceeds 10240,000 • Each of 2000 256 bytes in the 512Kbytes flash are expected to be written exactly 5000 times if absolutely uniform wear leveling is achieved

22. Conclusion • Provides complete ,efficient and reliable file system for micro sensor nodes • ELF is tailored to the resource constrained sensor nodes • Support for simple garbage collection • Optional best effort crash recovery mechanism • Achieves wear leveling • Allows random access of data