1 / 6

Quality Assurance Testing

Quality Assurance Testing. Tony Mack PlanetLab Developers Meeting. Overview. How do we validate/test our software? Developers test their own work, submit when they feel its ready. Does it build? Developers have different testing methods/processes (unit tests, manual tests, scripts, etc...)

alina
Download Presentation

Quality Assurance Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality Assurance Testing Tony MackPlanetLab Developers Meeting

  2. Overview • How do we validate/test our software? • Developers test their own work, submit when they feel its ready. • Does it build? • Developers have different testing methods/processes (unit tests, manual tests, scripts, etc...) • end to end system tests •  Did we test everything? • Backwards compatible? • Did new development break old functionality?

  3. Objectives • Extensible testing system (test harness) • Define a testing process (what are we testing? How do we test it?) • Configurable (Adding/Modifying/Updating test process should be easy) • Flexible (Test harness can run in different environments) • Automated • Developers can integrate their tests into the system • Accounting • Test results are logged. Can be reviewed at any time  • Track long term debugging efforts/progress

  4. Test Types • New development • Bug fixes • Backwards compatibility • Are we losing required functionality by upgrading/replacing/deprecating • Does current plc software support new node software? • Does new plc software support old node software? • Unit tests • Fine grain module testing • End to end tests • Do previous operations still work (can we create a slice at plc and log into it on a node) • Long term tests • Bugs that arise over time • Stress tests • What are the limits? How much of a beating can it take?

  5. Current Test Harness • Create a vserver and setup the testing environment (Optional) • Download and install the nightly MyPLC (if build failed and no nightly exists, abandon or test in a previously working environment) • Configure the PLC and add relevant test data to db • Site, Nodes/People/Slices at site, Slices on nodes, Keys for person accounts, NodeGroups, etc... • Run the api uint test(check if all api methods operate as expected and return what expected) • Attempt to boot selected nodes (using qemu) • Attempt to boot a 4.2 and 4.1 • If nodes come up in boot bootstate, check if NodeManager is running and creating slices • Execute node test scripts (cpu scheduler, planetflow, ...)

  6. To Do • Long term tests • Upgrading a PLC thats already installed • Specific NodeManger tests (slice attributes and resource management) • Multiple PLC's and federation • Hardware diversity found in real nodes

More Related