1 / 44

Testing in Salesforce

Testing in Salesforce. Going beyond 75% (or even 99%) statement coverage. Laurent Poulain lpoulain@progress.com. Disclaimer. This is NOT an exhaustive presentation about testing Great FREE online courses on testing Udacity : Software Testing

simeon
Download Presentation

Testing in Salesforce

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing in Salesforce Going beyond 75% (or even 99%) statement coverage • Laurent Poulain • lpoulain@progress.com

  2. Disclaimer • This is NOT an exhaustive presentation about testing • Great FREE online courses on testing • Udacity: Software Testing • edX: CS169.1x / CS169.2x: Engineering Software as a Service • Two developers, three opinions

  3. Why testing? • Because Salesforce requires it? • Because it’s a good practice • Useful for tedious manual tests • Useful after refactoring some code • “This code has been working fine for years. No need to retest it.” Yeah right! • Testing is a key component if you want to pass the Advanced Developer Certification

  4. One of the most expensive bugs in history • The Ariane 5 rocket launch (1996) • A software exception occurred in legacy code from Ariane 4 • An integer conversion turned awry in the stabilization subsystem

  5. One of the most expensive bugs in history • The Ariane 5 rocket launch (1996) • A software exception occurred in legacy code from Ariane 4 • An integer conversion turned awry in the stabilization subsystem • Cost (rocket + satellite): $370-$500 million • A simple test of the function would have detected the bug

  6. Morale of the story: even stable code can break

  7. Morale of the story: even stable code can break • Changes other than data can affect your code and/or your test • Triggers • Validation rules • Workflows • New required fields • …

  8. The various types of tests

  9. Searching for a needle in a haystack • A test suite can ONLY prove that your code has a bug • You cannot run all the tests possible, so you have to make some choices

  10. Code coverage • Statement coverage (used by Salesforce)

  11. Other types of code coverage • Method coverage (S0): are all the methods called? • Method coverage (S1): are all the methods called from all the places where they can be called? • Branch coverage (C1): for each branch, are the tests covering both scenarios where the condition is met and is not met?

  12. Code coverage example if (condition) { // block 1 ... } // block 2 ...

  13. Statement coverage if condition == true: 100% if (condition) { // block 1 ... } // block 2 ...

  14. Branch coverage if condition == true: 50% if (condition) { // block 1 ... } // block 2 ...

  15. Branch coverage if condition == true: 50% (in theory) if (isAdmin) { // block 1 ... } // block 2 ...

  16. Branch coverage+ if (isAdmin || isManager || isSuperUser) { // block 1 ... } // block 2 ...

  17. Other types of code coverage (ctn’d) • Path coverage (C2): are all the possible paths through the code executed? • Loop coverage: cover every loop with a scenario with 0, 1 and multiple iterations • MC/DC (modified condition/decision coverage): for each condition, test all possible boolean combinations

  18. Are we done now ?

  19. Code coverage example try { if (isAdmin) { // block 1 ... } // block 2 ... // bug => crash for non-admin users } catch (Exception e) { // handles the exception }

  20. Is the code doing the right thing? • Use assertions in your test • Add assertions in your code to detect internal inconsistencies • Two approaches • Black box testing: given an input, do we get the expected output? • What if you enter invalid data? • White box testing: given an input, is the program behaving as expected? • On Salesforce, needs to add code for testing purpose only • Sometimes, looking at Limits.get<limit name>() is enough • Too many white box tests it might be an indication you need to refactor your code

  21. The need of oracles • An oracle can tell whether the code is doing the right thing or not • Weak oracle: exception in code • Medium oracle: assertion in code • Strong oracle • Compare results with an alternate implementation • Function inverse pair (e.g. save and load) • Null space transformation (feed two inputs that should be treated exactly the same)

  22. Are we done now ????

  23. Still not finished… • Start with acceptance tests • Decompose in unit tests • Integration testing: testing with other systems the app is interacting with • Use StaticResourceCalloutMock • Random/fuzz testing: sending random data or random input • You may not want to deploy such tests

  24. How much testing? • Cost/benefit tradeoff: is the testing worth the time? • Critical parts should be well-tested (e.g. triggers) • It is not unusual to have more lines of test than lines of code itself • SQLite 3.8 has ~84 thousand lines of C code

  25. How much testing? • Cost/benefit tradeoff: is the testing worth the time? • Critical parts should be well-tested (e.g. triggers) • It is not unusual to have more lines of test than lines of code itself • SQLite 3.8 has ~84 thousand lines of C code • SQLite tests: 91 million lines of code (x1084 !)

  26. Salesforce Recommendations • Use the default “seeAllData=false” whenever possible • Use a separate class for tests • Create separate factory classes that will generate your test data • e.g. implement a factory method to create n Opportunities with m Contact per Opportunity • Bulkify your tests by feeding your code with 200 records • Use test.startTest() and test.stopTest() to get two sets of governor limits

  27. Check the documentation

  28. Methodology

  29. Testing is not always considered as a “finishing touch”

  30. Test-Driven Development (TDD)Behavior-Driven Design (BDD) • Both methodologies put writing tests first, coding second • Test early, test often • BDD is about the behavior of the application • A black box form of TDD testing • A good way to think of some tests: each feature should be tested • TDD can be very convenient when debugging code which is modifying data

  31. Test Examples

  32. Testing a complex VisualForce page • A Scorecard written in VF+apex • Contains 30+ case metrics • Several filters, breakdowns and ways to display the data • Some data is precomputed, some is computed on the fly • The challenges: • Lots of possible user options increases risk of bugs • Detecting bugs in numbers • Four types of tests • Code coverage • White box testing • Black box testing • Random testing -> Exhaustive testing

  33. White box test • Check that the right function is called and the right dynamic SOQL is generated • Doesn’t care about the results • Use a class called SupportTest that acts as a “debug log” that the test can read • Stores data as a key / value pair • Use only static functions for simplicity • Available on https://github.com/lpoulain/DebugLog

  34. SupportTest class SupportTest.reset(); … SupportTest.addValue(’compute’, ’Precomputed (6 months)’); SupportTest.addValue(’SOQL’, soql_query); SupportTest.addValue(’error’, e.getMessage()+’ - ’+getStackTraceString()); … System.assert(SupportTest.valueEquals(’compute’, expectedValue)); System.assert(SupportTest.getNbValues(’compute’) == 1); error = SupportTest.getOldestValue(’error’); System.assert(error == null, error); soqlError = SupportTest.SOQLWhoseWHEREEquals(’SOQL’, SOQLFilters) System.assert(soqlError == ’’, soqlError);

  35. Black box test • Runs some scenarios and compares the result with hard-coded data • The test class contains all the necessary test data: • The raw data • Scenarios (user selection + expected output) • A VisualForce page to display the test data • Use the test as a form of documentation • When a new metric is added, the test is FIRST updated

  36. A VisualForce page that displays the raw data…

  37. … and the scenarios

  38. Random test turned exhaustive test • Use the drill down capability as an oracle • Compared numbers from the Scorecard with numbers returned by the drilldown • Started as random tests • Decided to transform it to an exhaustive test • It is now a batch job • It is not an official test anymore • I did find a bug this way

  39. BDD on Salesforce

  40. BDD on Salesforce: Pickle • A BDD tool inspired from Ruby tool Cucumber • Allows to write some tests in plain English (some apex required behind the scenes) • Available on Github (https://github.com/lpoulain/pickle) • You can use Cucumber + Selenium, but it runs outside the Salesforce testing framework

  41. Pickle Test Example Given the following Users:u1|Joe SmithGiven the following Accounts exist:Account Id|Account Namea2 |Foo Inc.Given the following Cases exist:Case Id|OwnerId|AccountId|Subject |Case Origin|Escalatedc3 |u1 |a2 |This is a test|Web |trueGiven I am on page "My VF Page" (MyClass) with parameters foo=24When I set "My Field" to "Random Value"and I click on "Update"Then "result" should contain "OK"and "numeric Field" should be >= "5"and "foo" should be equal to "24"and the query [SELECT Case Id, Subject FROM Case WHERE "Case Origin" = ’Web’] should return:c3|This is a test

  42. Web interface

  43. Questions?

More Related