SAML Conformance Sub-Group Report

1. SAML Conformance Sub-Group Report Bob Griffin Face-to-face meeting August 29, 2001

2. Topics • Status • BindingConformance issue • Browser partition issue • Testing optional elements • Building test clients • Interoperability testing

3. Status • Conformance Clause • Version 0.006 in SAML spec; no change from June F2F • Describes goals, validation/certification, partitions • Conformance Program • Version 0.005 available • Includes cut at test cases for Authentication Authority Partition, including traceability to requirements; other test cases tdb • Certification • Decision at June F2F to focus on validation for SAML V1 • Test suite • Detailed design to be included in test cases following F2F • Traceability of SAML to S2ML, AuthXML… • Prateek to do SAML/S2ML traceability • Darren to do SAML/AuthXML traceability • Dave O. to do SAML/ITML traceability

4. Issue: BindingConformance • “Should protocol bindings be the subject of conformance?” • Concerns raised in issue write-up: • Profile is “a complete specification of the SAML aspects of some use case” while binding is “a building block in one or more profiles”. • Specifying conformance in terms of protocol/bindings increases the number of test cases • Conformance claims based on protocol/binding would not necessarily mean interoperability. • Discussion • Current model of “conformance partitions” does not define conformance in terms of use cases, but in terms of system entities in the static domain model: (authentication authority, attribute authority, policy decision point, policy enforcement point) • Conformance claims by SAML implementation or application would be for one or more partitions; • Within partition, subsets further allowed • For each system entity, test cases related to profiles and to protocols/bindings are defined

5. Authentication Authority Partition Test Cases HTTP Web Server Profile(s) • Valid Authentication Assertion Produced • Valid Authentication Assertion Consumed (eg, from another domain)? • Valid Authentication Assertion Artifact Produced • Valid Authentication Assertion Artifact Consumed Authentication Request/Response Protocol: HTTP, SOAP bindings • Valid Authentication Assertion Produced • Valid Authentication Assertion Artifact Produced • Valid Authentication Assertion Artifact Consumed

6. Authentication Authority Partition Tests Test case 1-4: Web Server Profile Web Server Request for authentication assertion embedded in HTTP Authentication assertion returned, embedded in HTTP, and checked by test client for validity SAML Conformance Test Client Test case 5-16: Protocol/Binding SAML implementation or application Request for authentication assertion, bound to HTTP Authentication assertion returned via HTTP and checked by test client for validity

7. BindingConformance (cont’d) • Discussion: • Should conformance be tested in terms of use cases rather than entities in domain and producer-consumer models? • Are profiles equivalent to use cases and therefore the way conformance should be specified? • Does testing protocols/bindings test interoperability? • Are protocols specific enough so that conformance can be defined in terms of protocols rather than protocol + binding? • Recommendation: • For now, continue with “conformance partitions”. • Include behavior related to both profiles and protocol/bindings in conformance partition testing.. • Implementation or application claiming conformance to a given partition must specify which bindings it supports for the protocol(s) in that partition • Not all bindings have to be supported to claim conformance to that partition.

8. Issue: Browser Partition • Should there be a conformance partition for browser, comparable to those for authentication authority, attribute authority, PEP, PDP? • Discussion: • The other partitions represent implementations (eg. Multi-domain single-signon product) or applications (eg, application server as PEP accepting assertions) of SAML. • Insofar as unmodified browser is goal, no comparable implementation/application related to browser? • Will all SAML-specific behavior related to profiles be captured by the other partitions? • Recommendation: • Do not define browser partition. • Check that all SAML behavior is captured by other partitions

9. Optional Elements • Issue: Test cases have to include sub-cases based on optional elements, encryption/signing, etc. • Discussion • Test suite should validate that SAML implementation or application has correctly specified an extension, but should not test the extension itself • No expectation of having registry for SAML extensions • Recommendation: following F2F, develop sub-cases for variants and optional elements to explore this issue further.

10. Saml Conformance Test Client Design • Test client has to evaluate responses received from SAML implementation or application to determine validity of assertions and (in case of protocols) response messages • Validity can be determined in several ways: • Inspection of returned assertion and/or message • Submission of assertion and/or message to open-source reference implementation • Follow-on action based on assertion or message (eg, test succesful consuming of authentication assertion by requesting protecting resource)

11. Test Client Design - 2 Test case 1: Web Server Profile Web Server Request for authentication assertion embedded in HTTP header SAML Conformance Test Client Authentication assertion returned, embedded in HTTP header, and checked by test client for validity Alternative 3. Resource request submitted to SAML implementation, using authentication assertion SAML implementation or application SAML Reference Implementation Reference Assertion Alternative 2. Received authentication assertion compared to reference assertion Alternative 1. Received authentication assertion submitted to reference implementation in resource request

12. Test Client Design – 3 • Recommendation: investigate options in terms of detailed design of test cases, to determine how feasible each approach is across all test cases

13. Interoperability Testing • Several options: • Test implementations/applications against open-source reference implementation (eg, submit assertions to reference implementation PEP) • Cross-vendor testing in customer situations • Cross-vendor testing in independent lab, etc • Cross-vendor testing at vendor sites