1 / 29

Formal Testing of Web Content using TTCN-3

Formal Testing of Web Content using TTCN-3. By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa. Motivation. Web Applications (WA) are the most widespread target domain of software development in computer history.

tamatha
Download Presentation

Formal Testing of Web Content using TTCN-3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formal Testing of Web Content using TTCN-3 By Robert Probert, Bernard Stepien, Pulei Xiong University of Ottawa

  2. Motivation • Web Applications (WA) are the most widespread target domain of software development in computer history. • WA are error prone because mostly because anyone can develop them and they are mostly distributed. • Testing of WAs: manual or using dedicated tools or robots.

  3. Current state of researchon formal Web content testing • XML based formalism • Xiaoping Jia and Hongmin Liu, Rigorous and Automatic Testing of Web Applications • Sahil Thaker, Robust Testing for Web Applications

  4. Web content testing elements • Hyperlink checking, including recursive • Text fragment checking • Protocol checking • Table checking • Form checking • Formatting checking • Mirror sites content checking • Version checking (languages,etc…) • Invalid HTML syntax checking • …

  5. Specific needs of web content testing • TTCN has been originally designed for telecommunication protocols. • In Telecom, every single data element is relevant and needs to be coded and decoded to and from the abstract layer. • In Web application, only part of the content is testable and needs to be extracted because most of the web page content is either too large or unpredictable. • Thus, in web content testing, we test only invariants.

  6. DePaul U Formalism BNF TestSpec ::= TestSuite* TestSuite ::= TestCase* TestCase ::= TestStep + TestStep ::= Name Condition opt RequestSpec ResponseSpec TestStep* RequestSpec ::= URL RecursiveSpec opt HearderSpec* ParameterSpec* VariableDecl* ResponseSpec ::= StatusCode ContentType HearderSpec* VariableDecl* Predicate + RecursiveSpec ::= Depth Domain opt HeaderSpec ::= Name CDATA optional opt ParameterSpec ::= Name CDATA optional opt Condition ::= Predicate Predicate ::= not Predicate | Predicate and predicate | Predicate or Predicate | Predicate implies Predicate | ( forall VariableDecl + Predicate ) | ( exists VariableDecl + Predicate ) | SimplePredicate SimplePredicate ::= MatchPred | ContainPred | ComparePred MatchPred ::= SelectExp MatchOp ( RegExp | CDATA ) MatchOp ::= equals | contains | startsWith | endsWith ComparisonPred ::= Exp ComparisonOp Exp Exp ::= NumberExp | DateExp ComparisonOp ::= == | != | < | <= | > | >= ContainPred ::= HrefSpec | FormSpec HrefSpec ::= URL CDATA opt VariableDecl opt FormSpec ::= ActionSpec opt MethodSpec opt VariableDecl ::= Name SelectExp

  7. Text Fragment checkingDePaul U formalism <testcase name="Content check using predicates"> <teststep name="Content check step one"> <requesturl="http://www.cs.depaul.edu/program"/> <responsestatuscode="200"> <and> <or> <matchop="contains" regexp="false" select="/html/body" value="Undergraduate Degree"/> <match op="contains" regexp="false" select="/html/body" value="Bachelor Degree"/> </or> <match op="contains" regexp="true" select="/html/body” value="[M|m]aster [D|d]egree"/> </and> </response> </teststep> </testcase>

  8. Mapping DePaul U Formalism to TTCN-3 • Testsuite -> testsuite • Testcase -> testcase • Teststep -> function • Request -> send • Response -> receive • Match -> built-in TTCN-3 matching mech. • Regexp -> pattern • Value -> template

  9. Text Fragment checking TTCN-3types and templates template charstring program_page_web_url := "http://www.cs.depaul.edu/program"; type record content_response_type { charstring status, charstring content } template content_response_type content_response_pred := { status := "HTTP/1.1 200 OK", content := pattern "*<BODY * [Undergraduate Degree | Bachelor Degree] * [[M|m]aster [D|d]egree] * </BODY>*" }

  10. Text Fragment checking TTCN-3test case testcase Content_check_pred_TC() runs on MTCType system SystemType { var verdicttype theOverallVerdict; map(mtc:web_port, system:system_web_port); theOverallVerdict := CheckContent_pred(program_page_web_url, content_response_pred); setverdict(theOverallVerdict); }

  11. Text Fragment checking TTCN-3test logic function CheckContent_pred(charstring theURLLink, content_response_type theContent_response) runs on MTCType return verdicttype { var charstring theBadResponse; var verdicttype theFinalVerdict; web_port.send(theURLLink); alt { [] web_port.receive(theContent_response) { theFinalVerdict := pass; } [] web_port.receive(charstring:?) -> value theBadResponse { log("Bad Response: " & theBadResponse); theFinalVerdict := fail; } } return theFinalVerdict; }

  12. Contains concept and TTCN-3 pattern • The ‘and’ operator does not imply sequence. • Permutations of values must be specified in the TTCN-3 template using pattern. • Combinatorics explosion when the number of text fragments increases (np). • TTCN-3 should implement a concept of “contains” with logical expressions

  13. Evaluation of TTCN-3 advantages with text fragment checking • DePaul U has the contains concept. • TTCN-3 has better structuring of data sent or received. • TTCN-3 has the concept of alternative to structure exceptions, etc… • TTCN-3 is more flexible because new functionalities can be added both at the abstract and adapter level.

  14. Recursive hyperlink checkingDePaul formalism <testcase name="Link check"> <teststep name="link check 1"> <request url="http://www.cs.depaul.edu", recursive="true", recursivedepth="3"/> <response statuscode="200"/> </teststep> </testcase>

  15. Recursive hyperlink checkingTTCN-3 Solution: Test case invokes a recursive function and appropriate list type definitions. testcase Link_Check_TC() runs on MTCType system SystemType { ... theOverallVerdict := CheckChildLink(main_page_web_url, 1); setverdict(theOverallVerdict); } type record of charstring HrefListType; type record hyperlink_response_type { charstring status, HrefListTypehrefList } template hyperlink_response_type hyperlink_response :={ status := "HTTP/1.1 200 OK", hrefList := ? }

  16. Recursive hyperlink checkingTTCN-3 recursive function function CheckChildLink(charstring theURLLink, integer theDepth) runs on MTCType return verdicttype { web_port.send(theURLLink); web_port.receive(hyperlink_response) -> value theHyperlink_response; if(theDepth <= maxDepth) { var integer theNewDepth; var HrefListType theResponseHrefList :=theHyperlink_response.hrefList; var integer numOfLinks := sizeof(theResponseHrefList); var integer i; theNewDepth := theDepth + 1; for(i:=0; i < numOfLinks; i:=i+1) { oneVerdict := CheckChildLink(theResponseHrefList[i], theNewDepth); if(oneVerdict == fail) { theFinalVerdict := fail; } } } return theFinalVerdict; }

  17. Evaluation of TTCN-3 advantages with recursive link checking • TTCN-3 is more powerful because it allows more filtering on the recursion. For example, one would only recurse through the links to other web pages internal to the same company and ignore external links. • TTCN-3 allows more sophisticated recursion control than simple depth limits. • DePaul formalism is more compact but also more limited.

  18. Quantified predicate checkingDePaul U Formalism <testsuite> <testcase name="Content check using quantified predicates"> <teststep name="content check step one" > <request url="http://jordan.cs.depaul.edu/webtest/testhref.htm"> </request> <response> <forall> <varbale name="l" select="descendent::a/@href"/> <match op="startswith" select="$l" regexp="false" value="http://"/> </forall> </response> </teststep> </testcase> </testsuite>

  19. Mapping to TTCN-3 • Forall -> TTCN-3 for loop with bounds • StartsWith -> TTCN-3 substring • Select -> done in TTCN-3 CODEC

  20. Quantified predicate checkingTTCN-3 testcaseForall_protocol_check_TC(…) runs on MTCType system SystemType { web_port.send(theURLLink); web_port.receive(hyperlink_response) -> value theHyperlink_response; all_match := true; for(i:=0; i < numOfLinks; i:=i+1) { var HrefListType theResponseHrefList := theHyperlink_response.hrefList; if(substr(theResponseHrefList[i],0,theLength) != theProtocol) { all_match := false; log("link: " & theResponseHrefList[i] & " is not of protocol " & theProtocol); } } if(all_match) { setverdict(pass); } else { setverdict(fail); } }

  21. Remarks about the TTCN-3 solution • Matching is not achieved via the inherent TTCN-3 matching mechanism, but instead using traditional programming languages features (substr). This is mainly because it is impossible to match a list of unknown length. • There is no parametric solution. Two different select values would require a dedicated definition of types, templates and test cases for each select values.

  22. Our Recommendationparametric CoDec Some fields act as parameters passed to the CODEC rather than matching values coming from the CODEC type record myRecord { integer field_1, charstring field_2 codecparameter, charstring field_3 } template myRecord myTemplate := { field_1 := 5, field_2 := “href”, field_3 := “abcd” }

  23. Sahil Thaker WebDeveloppers ltdForm testing formalism <oraclescope = form> <form method=“get” target=“”> <input type=hidden name=“” value=“”> <input type=submit> </oracle>

  24. Form testing in TTCN-3type definitions type record FormResponseType { charstring status, FormType form } type record FormType { charstring name, charstring method, charstring action, InputListType inputList } type record InputType { charstring name, charstring type_input, integer size_input, charstring value_input } type record of InputTypeInputListType ;

  25. Form testing in TTCN-3template definitions template FormType theRegistrationForm_template := { name := "subscription_form", method := "POST", action := "http://www.someone.com/cgi-bin/processSubscription.pl", inputList := { {name :="firstname", type_input := "text", size_input := 20, value_input := ""} , {name :="lastname", type_input := "text", size_input := 35, value_input := ""} , {name :="city", type_input := "text", size_input := 40, value_input := ""}, {name :="send_me_emails", type_input := "checkbox", size_input := 0, value_input := ""} , {name :="wine_and_cheese", type_input := "checkbox", size_input := 0, value_input := ""} , {name :="", type_input := "submit", size_input := 0, value_input := "send"} , {name :="", type_input := "reset", size_input := 0, value_input := "Oooops!_Let_me_try_again!"} } }

  26. Form testing in TTCN-3parametric testcase definition testcase Form_Check_TC(charstring theURLLink, FormResponseType theFormResponse) runs on MTCType system SystemType { … web_port.send(theURLLink); alt { [] web_port.receive(theFormResponse) -> value theFormResponseTypeValue { log("the form " & theFormResponseTypeValue.form.name & " conforms to specification"); setverdict(pass); } [] web_port.receive(FormResponseType:?) { log("in the catch all receive !"); setverdict(fail); } } }

  27. Form testing in TTCN-3test control definition • Maximizing re-usability using: • Parametric test cases • Parametric data templates control { execute (Form_Check_TC(form_page_web_url, form_response(theRegistrationForm_template))); execute (Form_Check_TC(another_web_page_url, form_response(another_form_template))) }

  28. Advantages of TTCN-3 • Parametric test cases • Parametric templates • Structuring at all levels, types, templates, test cases, functions • Concept of control • Test configuration specification

  29. Conclusions • TTCN-3 is adequate for web content testing. • There is no need to create new formalism dedicated to web content testing. • Instead, it would be more beneficial to use TTCN-3 to develop generic web content testing tools mostly because of TTCN-3 parametric features.

More Related