CAP6135: Malware and Software Vulnerability Analysis Fuzzing Test Example Cliff Zou Spring 2013

1. CAP6135: Malware and Software Vulnerability Analysis Fuzzing Test ExampleCliff ZouSpring 2013

2. Objective • Explain basic fuzzing with concrete coding example • Explain how the vulnerable code in programming project 2 is derived • Introduce several useful techniques in doing the fuzzing test on project 2

3. Example Code • $ fuzzTest-target 200 “what is this?” 2 • Example code needs three inputs • Int, string, Int int inputInteger; /* global variable */ if (argc != 4){ fprintf(stderr, "fuzzTest needs 3 input parameters: int string int!\n"); exit(0); } sscanf(argv[1], "%d", &inputInteger); my_func(inputInteger, argv[2], argv[3]); • Subfun my_func() introduces 3 man-made bugs

4. Bug # 1: Integer Overflow int my_func(short argLen, char *str, char *divStr) { int denominator; float x; char buf[bufLen]; if (argLen != inputInteger) { fprintf(stderr, "Bug #1: integer overflow triggered\n"); foo = (void *)0xbfffffff; foo(argLen); /* trigger illegal instruction fault */ exit(1); • Int variable inputInteger changes to short • Overflow happens when inputInteger>32767 • foo() is a function pointer • Give it an arbitrary address will cause illegal memory reference for executing code

5. Bug # 2: buffer Overflow char buf[10]; if (strlen(str) > 10){ fprintf(stderr, "Bug #2: buffer overflow triggered. strlen=%d\n", strlen(str)); strcpy(buf, str); /* trigger segmentation fault or stack smashing , */ return 2; /*if overwriting return address, it will cause segmentation fault */ }

6. Bug #3: divide by zero int denominator; float x; sscanf(divStr, "%d", &denominator); if (denominator == 0){ fprintf(stderr, "Bug #3: division by zero triggered\n"); x = argLen / denominator; foo = (void *)0xbffbffff; foo(argLen); /* trigger illegal instruction fault */ }else x = argLen / denominator; return 0;

7. Fuzzer Outline • Generate inputs (random or follow rules) firstInt = rand()%50000; secondInt = rand() % 2; arraySize = rand() % 20; charArray = (char *) malloc(arraySize); for (j=0; j< arraySize; j++) charArray[j] = 'A'; charArray[arraySize-1] = NULL; • Generate execution command line sprintf(buffer, "./fuzzTest-target %d \"%s\" %d\n", firstInt, charArray, secondInt); free(charArray); /* must free memory for repeat testing! */

8. Fuzzer Outline • Execute target code ret = system(buffer); • Obtain target execution exit code wait(&status); retCode = WEXITSTATUS(ret); • Check abnormal exit code and record inputs that cause the abnormal if ( retCode == 128+11 || retCode ==128+6) /* segmentation fault (11) or Abort (6) */ { printf("retCode=%d ## Input: firstInt = %d, arraySize = %d, secondInt = %d\n", retCode, firstInt, arraySize, secondInt); fflush(stdout); /*make sure output is print out immediately ! */ } • Repeat from start in generating inputs

9. How to Record Fuzzing Result? • When abnormal happens, record down inputs that cause the abnormal • Record the corresponding abnormal message printout by target code • Unix OS I/O definition: • stdin (0), stdout (1), stderr (2) • I/O redirection: • $ Command < data.txt: let stdin get from file (instead of keyboard) • $ Command > output.txt: let stdout redirect to file • $ Command 2> error.txt: let stderr redirect to file • $ Command &> output.txt: let stdout and stderr redirect to file • For our example: • $./fuzzTest100 &> output.txt

10. Program Project 2 Introduction

11. Manual Read Sample.jpg File • To understand the jpeg file format and the project’s ‘sample.format’ description, you need a HEX Editor: • In Unix: use “$hexdump sample.jpg > hex.txt” • Each two-byte value is shown as ‘daff’ where the first byte is ‘ff’ and second byte is ‘da’ ! • A bit confusing on the byte order • HexEdit for Win: http://www.physics.ohio-state.edu/~prewett/hexedit/ • This program shows each byte value, so no confusion on big-endian or little-endian stuff. • You can use windows accessories “calculator” to translate between decimal and hexadecimal values • Use ‘programmer’ option in ‘view’ menu

12. One-Round Fuzzing Outline • In our fuzzer, we need to first read sample.jpg into a char buffer array • Then, modify the buffer (randomly or follows format rules) • Then, write the content of the buffer to test.jpg file. • Then, invoke jpegconv on test.jpg to do fuzz test

13. Read sample.jpg into Buffer char imageBuf[10000]; /*enough to hold sample.jpg */ int fSize; FILE *fin, *fout; fin = fopen(“./sample.jpg”, "rb"); fout = fopen(“./test.jpg”, “wb"); fseek(fin, 0, SEEK_END); /* set file pointer to the file end */ fSize=ftell(fin); /*get input file size */ fseek(fin, 0, SEEK_SET); /* rewind the pointer to the start of file fin */ fread(imageBuf, 1, fSize, fin); /* read byte stream of the file */ fclose(fin); /* then, modify imageBuf randomly, */ /* or follow jpeg format on the header section*/

14. Jpeg Header Format • Now the ‘sample.jpg’ is in the char array imageBuf[] • Check the ‘sample.format’ for the Jpeg format • For example: • imageBuf[0] = 0xff; imageBuf[1] = 0xd8;  SOI header • imageBuf[158]=0xff; imageBuf[159]=0xc0;  SOF header • imageBuf[609]=0xff; imageBuf[610]=0xda;  SOS header • Simple fuzzing: Mutation-based fuzzing • Only work on Jpeg Header section since all bugs are in here • You may only be able to find a few bugs in this way • Of course, trying millions of inputs may find all bugs if you are lucky • Advanced fuzzing: Protocol-aware fuzzing • Follow the guide in project description, modify format sections step-by-step • Modify different section could trigger different bugs

15. Write fuzzed image to test.jpg fwrite (imageBuf , 1, fSize, fout ); /* if you modified the image size, then use the new fSize */ fclose (fout); /* then, invoke jpegconv on test.jpg for testing */ • Note that the ‘test.jpg’ will only save the newest fuzzed file!

16. Save Fuzzed Input That Causes Bug int status, ret, retCode; int crashNum = 0; char fileName[20]; /* saved fuzzed image file name */ char comBuf[200]; /* save the command line string */ sprintf(comBuf, “./jpegconv -ppm -outfile foo.ppm test.jpg"); ret = system(buffer); wait(&status); retCode = WEXITSTATUS(ret); if (retCode == 139){ /* Segmentation fault for a bug */ crashNum ++; sprintf(fileName, “./crashed-%d.jpg”, crashNum); fout = fopen(fileName, “wb"); fwrite (imageBuf, 1, fSize, fout ); fclose (fout); }

17. Notes • Remember, do not save every fuzzed input into image files! • There is no enough disk space for that on Eustis! • You will still have multiple fuzzed image saved for the same bug. • You can find smart way to only save one copy for each bug. • When one or two bugs are repeatedly triggered • Try to modify image on other format sections • Mutate image file in different ways • Change to different values • random, negative, zero, upper-bound… • Change different number of bytes • Consecutive, randomly picked….

18. Unsolved Task • How to Match crashed-x.jpg to its bug ID? • Hint: Jpegconv uses stderr to print out “BUG X TRIGGERED” • I will leave this task to you

19. Notes • Do not directly copy code in the slide! • The quotation mark has been changed by Word! • How many runs should I do? • No. of fuzzed input files • No. of saved fuzzed image files • In order to not blow your disk space quota in Eustis • No. of different bugs found • Need your code to process stderr message • Your code needs to check if fopen() succeeds or not!

20. Working Environment • You can do this project on Eustis, or any Linux machine you set up • Make sure ‘jpegconv’ works on your computer (see project description) • You can use any programming langrage in Linux for the project • But your code must be able to run under Eustis for project submission • Eustis support: Perl, Java, C, Python, Sbcl • Your report must explain how we can run your code in Eustis!

21. Last Words • After this detailed explanation and coding, the project should be not hard • My own mutation-based fuzzer only contains less than 60 lines in C • Find two bugs in 1300 inputs • Protocol-aware fuzzer will be longer