1 / 57

Tracing regression bugs

Tracing regression bugs. Presented by Dor Nir. Outline. Problem introduction. Regression bug – definition. Industry tools. Proposed solution. Experimental results. Future work. Micronose corp. A big company founded by Nataly noseman 1998 – Version 1 of nosepad. (great success).

leighton-ace
Download Presentation

Tracing regression bugs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tracing regression bugs Presented by Dor Nir

  2. Outline • Problem introduction. • Regression bug – definition. • Industry tools. • Proposed solution. • Experimental results. • Future work.

  3. Micronose corp. • A big company founded by Nataly noseman • 1998 – Version 1 of nosepad. (great success)

  4. Nosepad version 1 voidSave(){ ... bDirty = false; } void Exit(){ if(IsDirty()) { if(MsgBox(“Save your noses?")) Save(); } CloseWindow(); } } class Nosepad { bool bDirty; void AddNose(){ ... bDirty = true; } void DeleteNose(){ ... bDirty = true; } bool IsDirty(){ return bDirty; }

  5. Nosepad version 2 • New features requires • … • … • Undo/Redo mechanism. • … • Micronose expanding • Promotions. • New recruitments.

  6. Undo/Redo Design • Undo stack – Each operation will be added to the undo stack. • Redo stack - When undo operation this operation will move to the redo stack. Key “a” Key “b” Undo Redo

  7. Nosepad version 2 void AddNose(){ ... undoStack.Add(AddNoseOp); redoStack.Clear(); } void DeleteNose() { ... undoStack.Add(DelNoseOp); redoStack.Clear(); } . . . } class Nosepad { … Stack undoStack; Stack redoStack; void Undo(){ undoStack.Top().Operate(false); redoStack.Push(undoStack.Pop()); } void Redo(){ redoStack.Top().Operate(true); undoStack.Push(redoStack.Pop()); }

  8. Zelda from QA

  9. Nosepad version 2 correction void AddNose(){ ... undoStack.Add(AddNoseOp); redoStack.Clear(); } void DeleteNose() { undoStack.Add(DelNoseOp); redoStack.Clear(); } bool IsDirty(){ return bDirty & !undoStack.IsEmpty(); } class Nosepad {… Stack undoStack; Stack redoStack; void Undo(){ undoStack.Top().Operate(false); redoStack.Push(undoStack.Pop()); } void Redo(){ redoStack.Top().Operate(true); undoStack.Push(redoStack.Pop()); }

  10. Zelda from QA

  11. Regression bug observations • The second bug is a regression bug. • The same test will succeeded on version 1 and fail on version 2. • The specifications for version 1 haven't change for version 2 (only addition)

  12. Regression bug • Specifications • X • Y • Z • Specifications • X • Y • Z • A • B Version 1 Changes in code Version 2 Bug… But no regression

  13. Regression bug definition • Regression bug – Changes in existing code that change the behavior of the application so it does not meet a specification that was previously met.

  14. How to avoid regression bugs? • Prevent inserting regression bug to the code: • Simple design. • Programming language. • Good programming. • Methodology. • Test driven development. • Code review. • Communication. • Find regression bugs before release of the product. • Extensive testing. • White box \ Black box testing.

  15. Automatic tools • Find whether a regression bug exist. • Quick Test Professional.

  16. Where is it? • What was the cause for the regression bug? • What was the change that caused the regression bug?

  17. What is a change? • Change existing code lines • Adding new code lines. • Delete code lines.

  18. Problem definition • When getting a check point C that failed, and source code S of the AUT. We want to find the places (changes) in the code S that causes C to fail. • We want to do it independently of the source code language or technology. • We know that at time T (previous to the failure) the checkpoint passed.

  19. Solution 1 Programmer Source code QA Cooperation Tests

  20. Solution 1 - map Tests Source code Check text in message box File t.xml was Created successfully “SELECT NAMES from Table1” is not empty

  21. Solution 1 • Much work has to be done for each new test. • Maintenance is hard. • We end up with a lot of code to be analyzed. • Could use automatic tools (profilers).

  22. Solution 2 Check text in message box File t.xml was Created successfully Only changes “SELECT NAMES from Table1” is not empty

  23. Source Control • Version control tool. • Data base of source code. • Check-in \ Check-out operation. • History of versions. • Differences between versions. • Very common in software development. • Currently in market:VSS, starTeam, clear case, cvs and many more.

  24. Finding regression bug Check point to code tool Source Control Tool Change A Change B … Out put: Relevant changes: 1. Change X 2. Change Y 3. Change Z … Heuristics Input Failed check point First phase Second phase

  25. Heuristics (second phase) • Rank changes. • Each heuristic will get different weight. • Two kinds of heuristics: • Technology dependence. • Non technology dependence. 1 2 3

  26. Non-technology heuristics • Do not depend on the technology of the code. • Textual driven. • No semantics.

  27. Code lines affinity Check point Select "clerk 1" from the clerk tree (clerk number 2). Go to the next clerk. The next clerk is "clerk 3"

  28. Check in comment affinity Check-in comment Go to thenext waiter when next item event is raise Check point Select "clerk 1" from the clerk tree (clerk number 2). Go to the next clerk. The next clerk is "clerk 3"

  29. File affinity Words histogram in file Clerk.cpp Check point Select "clerk 1" from the clerk tree (clerk number 2). Go to the next clerk. The next clerk is "clerk 3"

  30. File Name affinity ClerkDlg.cpp Check point Select "clerk 1" from the clerk tree (clerk number 2). Go to the next clerk. The next clerk is "clerk 3"

  31. More possible non technology heuristics • Programmers history. • Reliable vs. “Prone to error” programmers. • Experience in the specific module. • Time of change. • Late at night. • Close to release dead line.

  32. Technology heuristics • Depend on the source code language. • Take advantage of known keywords. • Use the semantics.

  33. Function\Class\Namespace affinity Check point Select "clerk 1" from the clerk tree (clerk number 2). Go to the next clerk. The next clerk is "clerk 3"

  34. Code complexity • Deepness, number of branching. if(b1 && b2) { if(c2 && d1) c1 = true; else { if((c2 && d2) || e1) c1 = false; } } if(b1 && b2 && c2 && d1) c1 = true; >

  35. Words affinity problem rain, green, red, coat red, flower, white, black, cloud

  36. Words affinity problem (cont.) red, flower, white, black, cloud red, flower, white, black, cloud > rain, green, red, coat train, table, love

  37. Word affinity red red red < < flower blue red

  38. How can we measure affinity? • Vector space model of information retrieval.- Wong S.K.M , raghavan • Similarity of documents. • Improving web search results using affinity graph - benyu Zhang, Hau Li, Lei Ji, Wensi Xi, Weiguo Fan. • Similarity of documents. • Diversity vs. Information richness of documents.

  39. Affinity definition • Synonym (a) - Group of words that are synonyms of a or similar in the meaning to a. chosen, picked out; choice, superior, prime; discriminating, choosy, picky , select, selection = Synonym (choose)

  40. Words affinity definition (cont) 1 if a == b • ShallowAffinity (a,b) = 0 else 1 if a == b • Affinity (a,b) = else ShallowAffinity (synonym (a), synonym (b))

  41. Affinity of groups of words

  42. Using affinity in the tool • Words (C) = the group of words in the description of the checkpoint C. • Words (P) = Group of words in the source code/checkin/file etc…

  43. Using affinity in heuristics • Code line affinity: • Words (P, L) = Group of words in the source code located L lines from the change P. • β – coefficient that gives different weight for lines inside the change. • Check-in comment affinity:

  44. Using affinity in heuristics (cont.) • File affinity:P is a change in file Fwith histogram map.

  45. Using affinity in heuristics (cont.) • File name affinity: • Code elements affinity:

  46. Algorithm Input: C – Checkpoint. T – The last time checkpoint C passed. • Get the latest version of the source code for C from the source control tool. • Get files versions from the source control tool one by one until the version check-in time is smaller then T. For each file version: • Get the change between the two sequential versions. • Analyze and rank the change in respect to the checkpoint C (Rank(C,P)) • Add the rank to DB.

  47. Observations • and i ≠ j • Better affinity Better results • The project is not always in a valid state.

  48. Implementation • Visual source safe • Arexis merge – Diff tool. • MS Word • WordNet • MS Access – DB.

  49. WordNet • Developed at the University of Princtoen. • Large lexical database of English. • English nouns, verbs, adjectives, and adverbs are organized into synonym sets, each representing one underlying lexicalized concept. • Different relations link the synonym sets.

  50. Additional views • Group by file. • Group by time of change.

More Related