1 / 10

OMA-REQ-2003-0376-PoC_Usability_Test_Results PoC Usability Test Results

O P E N M O B I L E A L L I A N C E. OMA-REQ-2003-0376-PoC_Usability_Test_Results PoC Usability Test Results. Submitted To: Req Date: 31 May 2003 Availability: Public OMA Confidential Contact: Reijo Nousiainen, reijo.nousiainen@nokia.com Source: Nokia. X.

asis
Download Presentation

OMA-REQ-2003-0376-PoC_Usability_Test_Results PoC Usability Test Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. O P E N M O B I L E A L L I A N C E OMA-REQ-2003-0376-PoC_Usability_Test_ResultsPoC Usability Test Results Submitted To: Req Date: 31 May 2003 Availability: Public OMA Confidential Contact: Reijo Nousiainen, reijo.nousiainen@nokia.com Source: Nokia X This is an input to the Open Mobile Alliance. No assumption regarding use of this material should be made. OMA-Template-SlideCover-20030303 Slide #1

  2. Introduction to PoC –delay test • PoC Delay conversational tests were conducted by Nokia Research Center –Audio Listening Laboratory • Experiment was performed in Finnish (native) language, using 50 naïve test persons. In addition 5 persons with PMR experience attended the test. • Test facilities and arrangements were made according to ITU listening tests, and the test procedures were according to ITU test plans. • The user acceptance for Start to Speak and Turnaround times were tested by an application running in PC environment • MOS scale from 1 to 5 was used to give score to the conversation quality • The testing subject got a sentence on the display of PC and pressed the PTT (= space bar) and read the sentence aloud • The conversation party was substituted by pre-recorded sentences to minimize the possible sources of errors

  3. Abbreviations Used • StS (Start to Speak): • Time measured from the ptt button press to the indication (tone) of the permission to speak. • “Time You have to wait until starting to speak” • TaT (Turnaround Time): • Time measured from the ptt release to the moment, when the reply can be heard from own loudspeaker. • “Time You have to wait the answer for your question, statement, etc.” • CI (Confidence Interval, U=upper or L=lower limit) • Statistical T-test with 95% confidence limits.

  4. PoC –delay test • Test persons: • In experiment the panel consisted of 32 females and 18 males, where 35 persons were between age from 18 to 39 and 15 persons were over 40 years old. • Conditions: • There were 12 different conditions in the test with following StS and TAT (in sec). • Dialogues: • There were 10 different dialogues for test conditions and 2 for practice session. Each dialogue consisted of 3-4 sentences. • Randomization: • Each person performed the test in different pseudo-random order. • Each condition had same amount of 200 votes with 10 different dialogues.

  5. Test Conditions

  6. Quality scale • “How did the delays affect the fluencyof conversation:” 5. Not noticeably4. Noticeably, but not annoying3. Slightly annoying2. Annoying1. Very annoying

  7. Results from Q: “How did the delays affect the fluency of the conversation”

  8. Results from Q: “How did the delays affect the fluency of the conversation”

  9. Test Results Separated 1.5 3.2 4.0 MOS level 3 : ”Slightly annoying”

  10. Conclusions • The acceptable level of Start to Speak is about 3.2 sec for the consumer segment and about 1.5 sec for the PMR segment • The acceptable level of Turnaround Time is about 4 sec => acceptable TaT in the system is 2 ... 3 sec 1) 1) According to the experience the user needs 1 ... 2 sec for answering.

More Related