1 / 17

Bluetooth Interoperability Testing at ITTC

Bluetooth Interoperability Testing at ITTC. Dr. Joseph B. Evans Dr. Daniel Deavours Leon S. Searl Information & Telecommunication Technology Center. Overview. What is the Bluetooth SIG Why Test Interoperability Test Philosophy Types of Bluetooth Applications Tested

fawzia
Download Presentation

Bluetooth Interoperability Testing at ITTC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bluetooth Interoperability Testing at ITTC Dr. Joseph B. Evans Dr. Daniel Deavours Leon S. Searl Information & Telecommunication Technology Center

  2. Overview • What is the Bluetooth SIG • Why Test Interoperability • Test Philosophy • Types of Bluetooth Applications Tested • Types of Devices Tested • Bluetooth Interoperability Tests • Test Result Types • General Results So Far • Future Work

  3. Bluetooth Special Interest Group • The Bluetooth SIG is a trade association comprised of telecommunication, computing, and network industry companies that is driving the development of a low-cost short-range wireless specification for connection mobile products • Protocols – RF, Link Layer, etc. • Profiles (application APIs) – Serial Port, Object Push, Print, Audio, etc. • Promoters - 3COM, Agere, Ericsson, IBM, Intel, Microsoft, Motorola, Nokia, Toshiba, etc. • HQ in Overland Park, KS • Established in February, 1998 • First Specification release in July, 1999

  4. Why Do Interoperability Testing? • Bluetooth SIG would like to obtain snapshot of current state of interoperability of commercially shipped Bluetooth products • Assess the feasibility of having an Interoperability Laboratory • Determine if additional Bluetooth specifications are required

  5. Interoperability Test Philosophy • Test interoperability from user point of view • Did it work or fail? • How easy was it to use • User level debugging of failures • See if parameters are correctly set • Were user manual instructions followed? • User can’t tell which which device is responsible for Bluetooth related failure • Save all test data to SQL database for use by SIG

  6. Bluetooth Device Categories • Handheld (HH) • Mobile Phone (MP) • Personal Computer/USB/PCMCIA (PC) • Printer (PR) • Headset (HS) • Hands-free (HF) • Human Interface Device (HID)

  7. Interoperability Testing • Determine what Bluetooth operations are successful between pairs of Bluetooth capable devices • Business Card Transfer • Meeting Request Transfer • Memo Transfer • File Transfer • Send Fax • Dialup Networking • Data Synchronization • Print Object • Human Interface Device Input • Headset Audio

  8. Bluetooth Application Test Types HH-HH Bizcard, Meeting Request, Memo HH-PR Bizcard, Meeting Request MP-MP Bizcard, Meeting Request MP-PR Bizcard, Meeting Request MP-HS Audio, Voice Dial MP-HF Audio, Voice Dial PC-PC Bizcard, Meeting Request, File Exchange PC-PR Bizcard, Meeting Request, File PC-HH Bizcard, Meeting Request, File Exchange, Sync PC-MP Bizcard, Meeting Request, Dial-up Networking, Fax HH-MP Bizcard, Meeting Request, Dial-up Networking

  9. Bluetooth Application Test Types • All applications are tested using Public connections • An application from each device pair type test suite is chosen to test the following Bluetooth features for each pair of devices • Bonding • Secure Data

  10. Bluetooth Test Organization • Device Pair Suite • There is a Test Suite for each Device Type Pairing (Example HH-PR) • A Test Suite is made up of Step Groups • Step Groups • A Step Group exercises a Bluetooth Application (Bizcard, etc.) between two Bluetooth devices or initializes the devices • A Step Group is composed of individual Steps • Step • Instruction to a Test Operator to perform an application operation on one or both devices • Non-Bluetooth Setup: 1 device, no Step Result Data • Bluetooth Setup: 1 device, may have Step Result Data • Bluetooth Interoperation: 2 device, Step Result Data for Both Devices

  11. Example Test Organization • HH-MP Test Suite • Setup for other Step Groups … • Business Card Transfer Step Group • Create Bizcard on HH • Send Bizcard from HH to MP • Alter ‘name’ on Bizcard on MP • Send Bizcard from MP to HH • Remove all Bizcards from MP • Remove second Bizcard from HH • Meeting Request Step Group … • Bonded, Non-secure Step Group … • Public, Secure Step Group … • Dial-up Networking Step Group …

  12. Bluetooth Test Step Group Data • Device 1 ID • Device 2 ID • Test Operator Name • Start Date and Time • Unique Execution ID • Support of Device 1 for application tested by the Step Group • Support of Device 2 for application tested by the Step Group

  13. Bluetooth Test Step Measurements • Qualitative • Subjective ease of use (0-5 scale) • Ease of Use Comments for subjective scale < 3 • Step comment • Quantitative • Start date/time, Stop date/time • Step duration (seconds) • Action count (integer) • Connection failures/retries (integer) • Success/Failure (boolean) • Group execution ID • Unique step ID

  14. Measurement Summaries • Summary data includes the following • Number of device pairs tested • Number of measurement steps • Number of measurement step failures • Percent of step failures • Average subjective scale • Average connection failure loss • Summaries are produced for the following groupings • Total database • Suites • Step groups • Individual devices

  15. General Results • 20 devices tested • HF = 1 • HS = 4 • PR = 2 • MP = 2 • HH = 2 • PC = 7 of 10 • 44 device pairings tested (out of 75 possible) • 371 steps with measurements • most of the steps succeeded • 4.2 subjective usability (out of 5) of the steps that succeeded

  16. General Results • Terms used by various manufactures are inconsistent with each other • Examples: Business Card, Contact, Address • Inter-operability failure rates correspond to complexity of the user interface • HS & HF = 14.3% failure rate • 1 or 2 button interface • PC = 23.9 failure rate • Graphical user interface • Software, PC user interfaces are still in infancy

  17. Future Work • Refined interoperability measurements • Create well designed test application for test operator input (instead of current rapid prototype) • More specific interoperability data captured • Breakup subjective scale • Intuitive interface • User manual • Audio quality • Add new measures • Robustness (PC crashes) • Web-based test operator application • Multiple test operators to reduce measurement bias • Add standard deviation to test summaries in addition to mean

More Related