Services Provided by LCBO Quality Assurance

1. Services Provided by LCBO Quality Assurance VQA SENSORY EVALUATION METHODOLOGY Leading Sensory Evaluation Services

2. VQA Appellation System VQA Mandate • Appellation of Origin System • Quality Standard • Control of use of specified terms, descriptions and designations associated with the VQA appellation system

3. Role of Sensory Evaluation Testing • Sensory Quality • Chemical Composition • Microbiological Stability • Packaging and Labelling Standards Quality of Beverage Alcohol Products

4. Role of VQA Sensory Evaluation Testing • Free of technical faults / defects • Typicity of varietal character – for varietal wines • “A wine bearing varietal designation shall be assessed to determine if the varietal designation for the wine exhibits the predominant character of a wine produced from the designated grape variety/varieties” (VQA Rules) • Typicity of the wine category: Late Harvest, Icewine, Nouveau, Sparkling – Traditional Method, Icewine Dosage, Botrytized Wine (VQA Rules) Scope

5. Role of VQA Sensory Evaluation Testing • “A wine shall be deemed to have passed the taste test if a majority of the members of the Tasting Panel determine: • That the wine’s attributes fairly reflect the viticultural and oenological quality standards established in O.Reg. 406/00 (Rules) without defects or flaws; and • That the wine is representative of quality wines of the stated category” (VQA Rules)

6. Sensory Panels • Grading Panel – 4 panel groups of 5 panellists each • 28 Members (20 regular panellists, 8 alternate tasters)

7. Panel Member Selection Criteria • Panel members are LCBO Product Consultants who work in retail stores within metropolitan Toronto area • Continuous training/ development opportunities • LCBO Product Knowledge I, I, III • Certification from the Wine and Spirit Education Trust • Master of Wine Certification

8. Panel Member Selection Criteria • Panel members are subjected to annual : • Sensory evaluation testing • Product knowledge testing • 25 % of the questions are VQA specific • Demonstrated professionalism – Sensory Evaluations Code of Conduct

9. Panels Composition • Panel Groups Assembly Criteria: • Results from the annual testing • Testing performance history • Previous experience (as a Grading Panel member, from the industry) • Performance monitoring data • Sensory sensitivities Consistent Performance Reproducible Results Balanced Panels

10. Panel Training Training Elements • Aroma, flavour recognition • Product category recognition • Varietal character recognition • Regional character recognition • Winemaking techniques • Technological influences • Defect identification

11. Sample Presentation • Sample information: Varietal composition, vintage year, wine category, method of production (where applicable: sparkling wines, icewine dosage, etc) • Blind sample presentation: • Samples are presented without identifying markings • Uniform sample presentation

12. Sensory Evaluation Method Sensory Evaluation Grading System

13. Objectivity Applicability Efficiency Sensory Evaluation Grading System Objectives • Superior objectivity – system design eliminates bias • Wider system applicability– system design equally applies to all wine categories • Efficient and effective calculation of results – time savings, accuracy • Data Analysis Tools&Traceability of results – enabled by automated data collection and quantification of the results Client Satisfaction

14. Sensory Evaluation Grading System Sensory evaluation grading system elements: • New sensory evaluation grading method • Integrating technology: • Automated data collection • Automated data quantification, analysis and reporting

15. Sensory Evaluation Grading System • Appearance and Colour • Aroma (primary and secondary) and Bouquet • Taste • Harmony Sensory characteristics applicable to all types of beverage alcohol products. Appearance Harmony Colour Aroma Bouquet Taste

16. Sensory Evaluation Grading System Sensory characteristics and attributes are evaluated using five grading categories.

17. Grading Form

18. Integrated Sensory Evaluation Software Innovative Technology Data collection Data quantification Data analysis Data reporting Compusense Five

19. Data Collection – Tablet PCs • Automated • Interactive • Flexible-Wireless • Pen or/and keyboard • Handwritten notes

20. Grading Systems Comparison

21. QA of Sensory Evaluation Results Quality Sensory Evaluation Result Quality Assurance System Controls Verification Tasting Performance Monitoring Proficiency Programs

22. QA of Sensory Evaluation Results System Controls • VQA Tasting protocol (VQA Rules) • 2nd Bottle tasting • Grading System Controls: • Security of panellist registration • Forced completeness of the assessment • Ratings cannot be changed, once finalized • Sample evaluation cannot be redone, once completed • Validation questions in the grading questionnaire • Calculation validation based on statistical measures • Automated calculation and reporting

23. QA of Sensory Evaluation Results Verification Tasting • Verification Tasting - an independent sensory evaluation of the products in conditions identical to the tasting panel. • The verification process has a quality assurance role and provides a reference value that is then compared with the panel results to identify any discrepancies in the assessment. • Verifiers – qualified Quality Assurance tasters. A tasting session may require up 2 Verifiers. • Verifiers’results are typically not included in the calculation, unless discrepancy situation.

24. QA of Sensory Evaluation Results Panellists Performance Monitoring • Panellists’ performance is monitored for each tasting session • Measures: • Outliers Frequency, % - measure of rating accuracy • Rating Rank – measure of rating bias (high, low, trends) • Sensitivity data - missed defects, good detection, hyper-sensitivity • Feedback on performance – Feedback Report • Provides tools to help identify training opportunities

25. QA of Sensory Evaluation Results Panellist Feedback Report

26. QA of Sensory Evaluation Results Sensory Proficiency Programs • Internally designed to measure quality and consistency of the sensory evaluation assessments • Designed to measure: • System performance • Panel Performance • Panellist Performance • Typical measures: repeatability, reproducibility, bias, defects identification, etc. System Panel Panellist

27. Internal Proficiency Testing Repeatability

28. Internal Proficiency Testing Repeatability

29. Internal Proficiency Testing Reproducibility • No significant difference between reported results, i.e. median values (p-Value: 0.649671) at a 95% confidence level

30. Internal Proficiency Testing Reproducibility • No significant difference between panels' (i.e. all 5 panels) reported results (i.e. median value), at a 95% confidence level (p-value = 0.99605)

31. Sensory Evaluation Grading System • A superior method for performing sensory evaluations. • Eliminates many sources of bias. • Customized for use with all beverage alcohol products. • Flexibility (data collection, quantification & reporting). • Provides significant time savings in data management. • Provides tools to analyze panellist results to help identify training opportunities. Summary

32. LC Leading the Sensory Evaluation of Beverage Alcohol