WeBWorK and JUnit

1. WeBWorK and JUnit an automated code grading system

2. Summary • Introduction • JUnit basics • JUnit/WeBWorK extension • sample session • Future directions

3. Introduction • Want to make a system to • automatically grade small java assignments • usable from the very first day of class • little startup • provide appropriate feedback at a more accessible level than compiler errors

4. Automatic grading • Want a system to grade • program fragments • entire methods or combinations of methods • entire classes (or multiple classes) • Provide immediate feedback • decreases turnaround cycle • encourages exploration

5. An accessible system • Many students find the first several days in programming to be terrifying • new systems using different paradigms • programs that work differently • error messages that are undecipherable • have to write a lot of code just to get the simplest program

6. An accessible system • Our system is designed to • remove the development environment • allow program fragments • provide immediate feedback • Our system is not designed to replace the professor • but it will grant the student greater and more timely feedback

7. Provide appropriate feedback • Compilers check syntax not semantics • Exceptions catch only gross errors • Unit testing allows very specific testing to home in on what is correct and isn’t correct

8. JUnit basics • From www.junit.org • “JUnit is a simple framework to write repeatable tests.”

9. JUnit basics • As you write classes you write tests • for a LinkedList class • code to check for normal functioning of linked list • code to check exceptional circumstances • delete from empty, removing non-present element

10. JUnit architecture • tests are usually contained in separate class • LinkedList, LinkedListJUnit • JUnit will automatically run all methods that begin with test… • Test methods contain code to notify JUnit of failures • Failures may include not raising an exception • JUnit keeps track of success/failures

11. JUnit example • Imagine a Money class for keeping track of various international moneys • Constructor, add methods

12. JUnit test for Money public class MoneyJUnit extends TestCase { public void testSimpleAdd() {     Money m12CHF= new Money(12, "CHF"); Money m14CHF= new Money(14, "CHF");      Money expected= new Money(26, "CHF");      Money result= m12CHF.add(m14CHF);      assertTrue(expected.equals(result)); } }

13. Common JUnit methods • Assert.assertTrue(bool); • Assert.assertEquals(X , X); • Assert.assertNotNull(Object); • Assert.fail(); • All have optional messages to attach.

14. Handling exceptions public void testThrowsException() {try { Money m = new Money(10,”XYZ”); } catch (Exception IllegalTenderException ite) { // do nothing, it was supposed to do this return; } Assert.fail(); // shouldn’t get here } • Note uncaught exceptions fail by default.

15. JUnit and WeBWorK • Have students type in a class/method/program fragment • Insert the code into a template designed by the problem creator • Compile the code, reporting errors if relevant • Use the standard testing mechanisms to measure relative correctness of code

16. Webwork opening screen

17. JUnit opening screen (empty)

18. JUnit screen filled well

19. JUnit correct result

20. Webwork answer copied back

21. Webwork says it’s ok

22. Bad JUnit answer

23. JUnit agrees

24. Writing JUnit problems • Write a template class • Write a JUnit class to test it • Write a WeBWorK problem

25. public class ArraySort{ private int[] numList= {12, 10, 8, 6, 4}; public ArraySort(){ sort(); print(); } public void sort(){ replaceme } public void print(){ for(int i = 0; i<5; i++){ System.out.println( "" + numList[i]); } } public int[] getArray(){ return numList; } public static void main( String [] args){ ArraySort ast = new ArraySort(); } } Sample template class

26. import junit.framework.*; public class ArraySortJUnitTest extends TestCase { private int[] startArray = {12, 10, 8, 6, 4}; private ArraySort ast = new ArraySort(); private int[] endArray = ast.getArray(); public void testSlotOne(){ Assert.assertEquals(startArray[4], endArray[0]); } public void testSlotTwo(){ Assert.assertEquals( startArray[3], endArray[1]); } public void testSlotThree(){ Assert.assertEquals(startArray[2], endArray[2]); } public void testSlotFour(){ Assert.assertEquals(startArray[1],endArray[3]); } public void testSlotFive(){ Assert.assertEquals(startArray[0],endArray[4]); } public static Test suite(){ return new TestSuite(ArraySortJUnitTest.class); } public static void main(String[] args){ ArraySortJUnitTest ajst = new ArraySortJUnitTest(); } } Sample JUnit test

27. System Architecture • WeBWorK displays background info • User clicks on link which runs traditional Perl/CGI script • Displays exact question • Collects user input

28. System Architecture (2) • On user input, X.java template expanded • Expanded X.java and XJUnit.java copied to unique temp directory • Code compiled, errors reported back to user

29. System Architecture (3) • JUnit tests run on code • Code run inside strict Java sandbox • Not currently CPU Limited • Number of JUnit tests passed translated into score [0.0,1.0] (0.1 increments). • On user click, passed back to WeBWorK which records score.

30. Sample JUnitWebWork problem • Currently involves copying lots of code to setup transfer between JUnit and CGI-Perl • Will be greatly simplified • Elements edited by problem writer • Problem description • Name of template class

31. Next steps • Writing problems • Cleaning/testing code • Improving interface with WeBWorK • Want to remove the extra clicks • Use WeBWorK 2.0 • Provide Web interface for editing • Increasing security • Infinite loops still problem • Issues with doing file i/o