1 / 16

Tcl/Tk guidelines for improved automated regressions - a case-study

Tcl/Tk guidelines for improved automated regressions - a case-study. 17'th Annual Tcl/Tk Conference (Tcl'2010) October 11 - October 15, 2010. Saurabh Khaitan Madhur Bhatia Tushar Gupta. Agenda. Need for automated GUI regressions Automated GUI testing tools Guidelines for developers

aaron
Download Presentation

Tcl/Tk guidelines for improved automated regressions - a case-study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tcl/Tk guidelines for improved automated regressions - a case-study 17'th Annual Tcl/Tk Conference (Tcl'2010) October 11 - October 15, 2010 Saurabh Khaitan Madhur Bhatia Tushar Gupta

  2. Agenda • Need for automated GUI regressions • Automated GUI testing tools • Guidelines for developers • Testing strategies for QA engineers • Limitations and workarounds • Results

  3. Need for Automated GUI regressions • Growing complexity of GUI • Excessive permutation and combinations of sequence of steps to test manually. • Rule out “human error” in testing • Reduce time-to-market • Eliminates repeated testing efforts

  4. Automated GUI testing tools • Analog tools • Co-ordinate based tools • Tools –> T-Plan Robot, AutoIt • Limitations • Object based tools • Works on internal objects/widgets of the GUI • Tools ->Squish, TKReplay

  5. Guidelines for developers • Direct use of internal functions • Use of global arrays • Use of environment variables • Use of algorithmic functions • Text representation of graphical display

  6. Use of Internal functions and global arrays Internal Function call to create annotation window createAnnotationFileWin addNetToAnnotationWin top.AIn[1:0] addNetToAnnotationWin top.BIn addNetToAnnotationWin top.w1[9:6] addNetToAnnotationWin top.YOut addNetToAnnotationWin top.ZOut $annotationWin(fileTypeCB) invoke $annotationWin(fileTypeCB) selection set 1 $annotationWin(fileTypeCB) invoke saveAnnotationFile Internal Function call to Add net to annotation window Call to global arrays Internal Function call to save annotation file

  7. Use of algorithmic functions and environment variables proc findpath {fullpath separator tree} { # sanity checks … #tree search algorithm … # tree update … # code for regressions if {[info exists ::env(MED_REGRESSIONS)]} { if {$found} { set msg “path found” } else { set msg “path not found” } echo $msg } } Algorithmic function call findpath to perform complex GUI operation The function performs operations like tree search and tree update This part of code is enabled under an environment variable

  8. Text representation of Graphical display

  9. Testing strategies for QA engineers • Verification Points • Synchronization points • Use of global procedures • Offline debug • Enable debugging screenshots

  10. Verification Points # Verification Point 'newfile' test compare [property get [findObject ":vsim.dockbar.tbf0.standard.tb.button_0"] state] "normal" test compare [property get [findObject ":vsim.dockbar.tbf0.standard.tb.button_0"] image] "__new_icon" • Checks Inserted in the code to verify the state of the GUI • test compare [property get $widget $prop_name] $expected_value Name of the VP Checking the state and the image of the button widget using compare calls

  11. Synchronization points • What is SP • Types of Synchronization • Time Synchronization • Object Synchronization • No Synchronization • waitForObjectItem $objectname $itemtext waitForObjectItem ":vsim.#mBar" "File" invoke activateItem ":vsim.#mBar" "File" waitForObjectItem ":vsim.#mBar.#mBar#file" "New" invoke activateItem ":vsim.#mBar.#mBar#file" "New" waitForObjectItem ":vsim.#mBar.#mBar#file.#mBar#file#new" "Project..." invoke activateItem ":vsim.#mBar.#mBar#file.#mBar#file#new" "Project..." Synchronization Points inserted before every click

  12. Use of global procedures proc main {} { snooze 10 #sourcing all global scripts source [findFile scripts "clean.tcl"] source [findFile scripts "analyze.tcl"] source [findFile scripts "close.tcl"] # global procedure – clean_all clean_all # global procedure - analyze analyze #global procedure – close_proj close_proj } ` Sourcing global scripts Call to different procedures

  13. Limitations and Workarounds • Drag and Drop • Use of black box GUI testing tools • Custom widgets • Test using conventional analog tools • Support for custom widgets from automation tool • Canvas widgets • Screenshot Verification Points • Textual dump of graphical display

  14. Screenshot Verification Points Screenshot masks Canvas widget Hierarchical name

  15. Results • Automated and stable GUI testing regressions • Reduced testing time • Improved test coverage • Better product Quality – Customer reported bugs reduced significantly

  16. Thank You

More Related