Combinatorial Testing on ID3v2 Tags of MP3 Files

1. Combinatorial Testing onID3v2 Tags of MP3 Files Zhiqiang Zhang1, Xiaojian Liu2, Jian Zhang1 1 Institute of Software, Chinese Academy of Sciences 2 Institute of Automation, Shandong Academy of Sciences

2. Introduction • MP3 (MPEG-1 or MPEG-2 audio layer III) is an audio file format developed by the Moving Pictures Experts Group (MPEG) • One of the most popular audio formats • Supported by almost all audio players • Usually, an MP3 file comes with an ID3 tag, which stores audio information such as: • Title, artist, album, … • ID3 has two unrelated versions: • ID3v1 & ID3v2

3. Overview • ID3v2 tag format • Test goals • Modeling • Experiments &Results • Conclusion

4. Combinatorial Testing • Considerably high fault coverage • Model the system under test (SUT) as a parameterized black box • Usually use a covering array as the test set • Small # of test cases

5. MP3 frames& ID3v2 tags • An MP3 file is built up from a sequence of MP3 frames (MPEG frames) • A short segment of audio data • The first 11 or 12 bits are always set, can be used for synchronization • Sometimes MP3 frames depend on each other, and cannot be freely cut • ID3v2 tag: • A metadata container for audio file information • Located at the beginning of an MP3 file

6. ID3v2 tag format • In our work, we consider a subset of the ID3v2 tag format • An ID3v2 header • Several ID3v2 frames • ID3v2 header format: ID3v2/file identifier “ID3” ID3v2 version $03 00 ID3v2 flags %abc00000 ID3v2 size 4 * %0xxxxxxx Tag size

7. ID3v2 Frames • An ID3v2 frame consists of a frame header and the frame content • Frame header Frame ID $xx xxxxxx (four characters) Size $xx xxxxxx Flags $xx xx • Text information frames Text encoding $xx Information <text string> Frame size

8. Text Encodings • Encodings commonly used in China: • $00: ASCII or GBK • $01: Unicode (UTF-16) • Little-endian (LE) • Big-endian (BE) • $02: Unicode BE • $03: UTF-8

9. Test Goals • Usually, tag processing and audio playing are performed by separate modules of MP3 players • Audio playing is often done by a decoder (out of the scope of our work) • ID3v2 tag processing • Goal I: text information recognition and display • Text encodings of some frames may influence the processing of other frames • Goal II: robustness against bad header & frame sizes • Offset related • May cause buffer overflow or vulnerable read operations

10. Modeling (Test Goal I) • Six types of text information frames • TIT2 (title), TPE1 (artist), TALB(album), TRCK (track number), TCON (genre) and TYER(year) • The model:

11. Modeling (Test Goal II) • The sizes indicated by the ID3v2 header & frame size bytes might be faulty • May cause critical failures • For building each test case • Set the header and frame size bytes • Set the actual header and frame sizes • Build an ID3v2 frame • Build an ID3v2 tag by filling the content with frames • Attach MP3 frames

12. Modeling (Test Goal II) • Building an ID3v2 frame • Building an ID3v2 tag Frame size bytes Tag size bytes Actual frame size Actual tag size

13. Modeling (Test Goal II) • Building a complete MP3 file • Attach a sequence of MP3 frames after the ID3v2 tag • In some cases, the actual tag size is not an integral multiple of the total frame size • To fill the remaining space: 4 filling modes

14. Filling Modes Tag size Tag size Tag size Tag size ID3v2 frame ID3v2 frame ID3v2 frame ID3v2 frame $00 00 00… ID3v2 frame ID3v2 frame MP3 frames MP3 frames MP3 frames MP3 frames Safe Overflow None Cut

15. Modeling (Test Goal II) • TS: tag size indicated by the ID3v2 header size bytes • TSD: difference of the actual tag size compared with TS • FS: frame size indicated by the frame header size bytes • FSD: difference of the actual frame size compared with FS • FM: filling mode • ATCHMP3: whether MP3 frames are attached or not

16. Modeling (Test Goal II) • Normal constraints • IF TS=0KB THEN TSD<>-1KB • IF FS=0KB THEN FSD<>-64B • When TS=0KB and TSD=0KB, the actual tag size is 0KB. So no frames will be filled. Thus FS, FSD and FM are invalid parameters • Introduce a special parameter value ‘#’ for invalid parameters • IF TS=0KB AND TSD=0KBTHEN FS=# AND FSD=# AND FM=# • IF TS<>0KB OR TSD<>0KB THENFS<># AND FSD<># AND FM<>#

17. Experiments • Test subjects: • I: an on-vehicle leisure and entertainment system • II: a portable MP3 player • Test generation: • Use Microsoft’s PICT • Test Goal I: 57 test cases (strength=2) • Test Goal II: 59 test cases (strength=3)

18. Experimental Results(Test Goal I, Subject I) • 2 passed, 55 failed • Actually, only ASCII text can be displayed

19. Experimental Results(Test Goal I, Subject II) • 33 passed, 24 failed • Problem when processing two encodings • Failures caused by 1-2 parameters

20. Experimental Results(Test Goal II, Subject I) • 52 passed, 7 failed • Failures caused by 4 parameters

21. Experimental Results(Test Goal II, Subject II) • All test cases passed • We occasionally found a bug • When ATCHMP3=NO, the subject • Displays that the file is damaged for a few seconds • Proceeds to the next file. • Just between this two moments, it is not possible to play back to the previous file • Not Discovered by CT!!

22. Conclusion • Benefit from CT: • Small test suite • Test Goal I: 57 out of 7323=2744 exhaustive test cases • Test Goal II: 59 out of 3442=648 exhaustive test cases (including test cases not satisfying the constraints) • Interaction faults discovered • Failures are caused by 1-4 parameters

23. Additional Results • Experiments are conducted on two PC audio players • PC audio player I perfectly passes all the tests • PC audio player II: • Test Goal I: similar failures as subject I • Test Goal II: • When all test cases are added to the player playlist, the player’s UI begin to respond slowly • The player got stuck when playing 4 test cases

24. Thank you！