Assessment Platform Development Recommendations

1. Assessment Platform Development Recommendations Building the Next Generation Assessment Platform – the Consortia Opportunity January 2011

2. Contents • Background • Assessment Platform Opportunity • Potential Project Management Approach • Structure • Advisory Board • Potential Service Offerings / Consortia Needs • Conclusion • Appendix – Advisory Board and Team Bios www.assessmentgroup.org

3. Important changes are on the way in Education “I am calling on our nation‘s Governors and state education chiefs to develop standards and assessments that don‘t simply measure whether students can fill in a bubble on a test, but whether they possess 21st century skills like problem-solving and critical thinking, entrepreneurship and creativity.” -President Barack Obama, March 10, 2009 “Our education system at all levels will leverage the power of technology to measure what matters and use assessment data for continuous improvement.” - National Educational Technology Plan 2010 (Draft), U.S. Department of Education “This new generation of state assessments will be an absolute game-changer in public education . . . For the first time, many teachers will have the assessments they have longed for.” – Arne Duncan www.assessmentgroup.org

4. … and present opportunities to have historic impacts on the success of U.S. Education These changes will have significant impacts on the current “system”… Legislation / RTTT / Federal Funding Children Learning Experiences Curriculum / Assessments 21st Century Educ. System District / School / Teacher / Student Trends Education Eco-System Educator / Administration Practices Technologies in Schools CCSS / Marketplace Trends / Innovations / Technology www.assessmentgroup.org

5. The consortia will play a key role in driving a transformation in Education • Consortia will help 100% of their member states implement new assessments by 2014/2015 • The leadership role of the consortia will help drive: • A new generation of assessments • Aligned to the Common Core standards • High quality • Iterative with instruction/curriculum • Test 21st century and other key skills (e.g., critical thinking, creativity) • Flexibly built for tomorrow – e.g., online, interactive • A comprehensive, flexible next generation assessment platform • Transition of stakeholders to new processes and technologies • Cost affordability and reliable implementation for all stakeholders www.assessmentgroup.org

6. Technology will be a central component of the Assessment transformation • Technology Characteristics • National scale – huge trxn. volumes / storage • Flexible to meet diverse state needs • Technology-enabled Capabilities • Linkage between curriculum and assessment • Flexible to deliver online (multi-device/interoperability) • Configurable for various assessment types/ purposes (summative, formative, adaptive) • New item/content types – e.g., simulations, graphics, games • Content sharing/repurposing • Scoring - near real-time; AI; distributed • Enhanced reporting for all stakeholders • Linkage to Education communities of interest Next Generation Assessment Platform Next Generation Education Platform www.assessmentgroup.org

7. New technology trends can benefit development of the new Platform • Open system/open source approach • Leverage existing “best of breed” technology solutions / vendors (including tying into “clouds” of major software providers) • Source content openly and collaborate with a broad variety of industries and disciplines (e.g., crowd sourcing) • Continual maturing/refresh of the “system” over time • “App Store” / Software as a Service (SaaS) approach (potential ongoing revenue source for the consortia) • Device independence – mobile, tablet, PC, e-blackboards • Maturing of learning management systems • Large-scale data management • New hosting/infrastructure models (vendor, university, consortia) www.assessmentgroup.org

8. States have some resistance to implementing assessment technology due to the risks / costs Change will not come easy given the current state of online assessment • Most states still administer assessments in PPT mode • Success with online testing has been mixed • State and district technology resources are very thin • Implementation costs have been unpredictable and high • A comprehensive “how to implement online testing” set of guidelines does not exist • Cost savings from similar efforts have not met expectations www.assessmentgroup.org

9. Success on this complex effort can be achieved by leveraging proven methods… Key Success Factors • Involve Education’s best and brightest early in the process • Get clear buy-in and consensus from all stakeholders early with respect to platform “business requirements” • Learn from others’ Assessment program mistakes • Leverage successes in other Education transformations • Approaches – timeframes, resources, key success factors • Technologies – leverage what already exists in the market today • Stage implementation to help States be successful • Share know-how/resources where possible to ensure the right support • Develop a transparent, well-understood, reliable cost model up front • Anticipate the needs of the future and build in flexibility • Employ the right resources and processes to help get there www.assessmentgroup.org

10. ASG brings the expertise to support the Consortia’s coordination and delivery of the Platform And by following a structured approach Next Generation Assessment Platform Strategy / Planning Design Implementation / Rollout www.assessmentgroup.org

11. Recommended Management Approach www.assessmentgroup.org

12. A combination of Education and Technology skillsets/expertise will be needed • Educational Expertise • Assessment / measurement • Standards (common core, 21st century, etc.) • Education trends • Classroom experience • School system experience • Administration experience • School data systems • School technology systems • Education trends • Business / Tech. Expertise • Strategy / Planning • Ed Technology Trends • Technology Innovations • Software Industry experience • Program Management • Requirements / Design • Application Development • Architecture • Infrastructure www.assessmentgroup.org

13. The skills required can be effectively provided be an external partner… • Risk of failure is too high if the needed skill are not obtained and proper business requirements are not developed • Combination of skillsets and focused attention needed to manage the process is likely best procured outside the consortia due to consortia resource constraints • A neutral 3rd party (non assessment platform vendor, non technology developer) should be considered to manage the process • Key vendor characteristics • Familiar with both technology and educational assessment • Able to leverage technology resources as needed • Able to bring the right business, management and technology skills • Experience in managing similar projects with a proven methodology www.assessmentgroup.org

14. … that can bring the right skills at the right time • Educational Expertise • Assessment / measurement • Standards (common core, 21st century, etc.) • Education trends • Classroom experience • School system experience • Administration experience • Education trends • Business / Tech. Expertise • Strategy / Planning • Ed Technology Trends • Technology Innovations • Software Industry experience • Program Management • Requirements / Design • Application Development • Architecture • Infrastructure • ASG recommends a dynamic Staffing Model: • Core team of key staff involved throughout for oversight and continuity • Individual representatives will rotate on and off the team based on the skills required for that phase • Consultants hired to fill short term skill gaps • There are significant benefits to this approach • Enables the consortia to get the most qualified people and expertise • Minimizes overhead, reduces costs and results in the best product Technical / Tactical / Implementation Skills End Start Strategic / Planning / Business Acumen Key Skills Required • Strategy • Stakeholder Alignment • Planning • Innovation • Design • Technical Evaluation • Development • Integration • Test / Rollout • Change Mgmt www.assessmentgroup.org

15. An Advisory Board of key experts will ensure the best solution is delivered • Role of the Advisory Board: • Provide Education industry and/or technology innovation insight and subject matter expertise • Assist in aligning key stakeholders • Oversee the progress / direction of the overall initiative and provide support to the Consortia • Constituents: • Senior education technology experts from world class software and hardware companies • Representatives from Cisco, Microsoft, IBM, Google, Booz Allen, etc. • These companies all have focused efforts in the Education space • All see the greater value in improving the current educational system in the US www.assessmentgroup.org

16. A Powerful Team Advisory Board - Education Leaders from Top Software/Technology Companies CSCO Booz Other MSFT GOOG IBM Strategic Technologists Next Generation Assessment System Database Architecture Core Technology Team Program Management

17. Advisory Board members • Charles Fadel, Global Education Lead, Cisco Systems • Cisco board member at Partnership for 21st Century Skills, Innovate/Educate and Change the Equation • Vice-chair of the Education committee of the Business and Industry Advisory Committee (BIAC) to the Organization for Economic Co-operation and Development (OECD) • Worked on education projects with more than thirty countries and states • Visiting Scholar at Wharton/UPenn where he recently taught a class on Technologies for Learning • Co-authored 21st Century Skills: Learning for Life in our Times • Recently served on the Massachusetts Governor’s Readiness Project as well as its 21st Century Skills task force • Visiting Practitioner, Harvard Graduate School of Education www.assessmentgroup.org

18. Advisory Board members • Hans Van Delden, VP Technology Practice, Booz and Company • Specializes in the strategy, operations and technology for educational assessment businesses • Led 10+ strategy, operations and technology programs for several education companies including: • Technology Strategy / Architecture for an Online Assessment solution (Educational Assessment Provider) – included an evaluation of the technology providers in the educational assessment market • Architecture Review (functional and technical) of custom-developed Online Assessment Platform (High-stakes State Educational Assessment Provider) • Assessment, Strategy and Roadmap for the Online Assessment Solutions (Educational Services company) • Experience working in 12 countries across North America, Europe and Asia • Earned five degrees from the Massachusetts Institute of Technology after five years of study • Twice awarded Booz & Company’s Professional Excellence Award, highest award for client service www.assessmentgroup.org

19. Advisory Board members • Steve Katz, Technology Strategist and Entrepreneur • Thought leader in technology innovation and strategy • Management roles in advancing innovation at IBM, Apple and Sony Pictures Entertainment • Executive strategy consultant to leading technology firms including Microsoft, Sony Corporation, SAP and AT&T • Serial entrepreneur; founded a number of pioneering ventures backed by Kleiner Perkins and other top VCs • Currently, founding partner in a strategy consulting boutique, O8 Partners - helps drive major partnerships and strategy for emerging technology companies • Currently entrepreneur-in-residence at MHS Capital, an early stage venture firm • Involved in several non-profit and philanthropic activities, including serving as a founding member of the Full Circle Fund (partners with outstanding social innovators to explore different approaches to Educational reform) www.assessmentgroup.org

20. Advisory Board Members • John Behrens, Director Networking Academy Learning Systems Development, Cisco, Inc. • Responsible for the design and development of the Networking Academy’s global on-line instructional offerings that integrate interactive curricular and assessment technologies with hands on activities. These curriculum, assessment, and gaming tools serve more than 700,000 students per semester in 165 countries across more than 10 languages. Over 100 million assessments have been delivered to date through the Cisco Networking Academy. • Dr. Behrens’ team integrates complex simulations in instruction and assessment and developed authoring tools to enable instructor-led visualization of abstract networking concepts. • John’s research interests focus on the intersection of cognitive, statistical and computing issues in instruction, assessment, and on-line learning. • Adjunct Assistant Research Professor, Department of Psychology, University of Notre Dame (Indiana, USA). www.assessmentgroup.org

21. Advisory Board members • Ed Roeber, Professor, College of Education, Michigan State University; Partner and Co-Founder, Assessment Solutions Group • John Olson, Partner and Co-Founder, Assessment Solutions Group • Barry Topol, Managing Partner and Co-Founder, Assessment Solutions Group (full bios on all advisory board members are located in the appendix) www.assessmentgroup.org

22. Potential core technology team members • Jay Whitchurch (Program Lead/Executive) • CEO and co-founder of campus2careers, an online career community that helps college students prepare for, find, and launch their careers • Senior Vice President, Educational Assessment (K-12), Pearson Education • Assessment portfolio included classroom-based (formative), skills (diagnostic/readiness), comparability (achievement), and reporting products/services • Chief Information Officer and Senior Vice President at Harcourt Assessment • Led a team of 150+ Technology professionals in: Strategy/ Planning, Architecture, Applications,  Infrastructure, Program Management, and Vendor Management • 13 years in management consulting driving strategic business transformations for global Healthcare, Education, Nonprofit, and Public Sector clients • Principal, Booz & Company, Strategy/Technology Practice – experience included working with a leading Education company to define an online strategy and associated business cases, priorities, required investments, and transition approach • Senior Consultant, Accenture – Led teams of client and Accenture staff on major technology change initiatives including the design and implementation of large-scale Public Sector systems www.assessmentgroup.org

23. Potential core technology team members • Technology Strategist / Innovator • Recognized as a thought leader on next-generation assessment systems • Over 20 years of technology leadership experience and over 6 years of experience in K-12 online assessment • Currently a Technology Director for a major assessment/education co. • Responsible for developing and maintaining the strategic direction and architectural road map for PK-12 online assessment • Leads the technology development for the current employer’s primary initiative around Common Core and the Consortia • Well established relationships with multiple online assessment companies and is considered a subject matter expert on the capabilities of various providers and publishers.   • Led the technology due diligence process for evaluating potential online assessment partners / acquisitions for two major Education companies • Provided technology leadership to multiple, established companies and held CTO/CIO positions in various entities during his career www.assessmentgroup.org

24. Potential core technology team members • Program Manager • Currently a lead in a top Technology company • 12+ years of experience in general management consulting for Fortune 500 clients • Focused on strategy based transformation for the Media & Technology industries • Strong experience in business / technology strategy, business case development, financial planning,  and “go-no-go” decision analysis • Demonstrated track record for leading and helping with the implementation of growth strategies and new product development • Led large transformational teams – delivering against tight timelines • Entrepreneur in the education space, owns two Montessori schools that were built ground up - concept to operational • Experienced in design and development of applications • Education: • MBA, Duke University, M.S. in Materials Science (Biomechanics) Michigan State University, M.S. in Mechanical Engineering (Robotics) Villanova www.assessmentgroup.org

25. Potential core technology team members • Milan Malkani (Architect / Technology lead) • Technologist and entrepreneur • Involved with the design and implementation of complex software systems for established companies (e.g., Dell, TheKnot.com) and startups (e.g., ActiveVendors.com and campus2careers.com) • Spent 3 years travelling the US and abroad to train the software development groups of IT departments for large organizations (e.g., Wells Fargo, US Navy and Army, MD Anderson, etc) • Proven track record of bringing applications to market in record time www.assessmentgroup.org

26. Potential ASGService Offerings www.assessmentgroup.org

27. ASG can support the Consortia across all key phases of the initiative Tasks by Project Phase Next Generation Assessment Platform Strategy / Planning Design Implementation / Rollout Initial Key Elements Requirements Definition Functional and Technical Design State Financial Models Technology Evaluation Build vs. Buy Systems Integration Plan Development RFP Preparation / Evaluation Rollout Plan Development Program Management www.assessmentgroup.org

28. The Strategy / Planning Phase will form the foundation for success Align Stakeholders Identify Process and Change Mgmt. Impacts • Stakeholder review/input sessions • Business Requirement iterations • Keep/start/stop activities • Transition planning Define/Confirm Common Core Assessment Vision Develop High-level Business Requirements Develop Plan for Design and Implementation • Roadmap for process, people, technology rollout and transitions • Top-down estimates on costs/timeframe • Priorities • Resource / skills requirements • Project launch • Stakeholder identification • Stakeholder interviews • Objectives / expectations • Constraints • “Must haves” • “Nice to haves” • Out-of-scope Develop High-level Technology Requirements Survey Current Technology Marketplace • Application • Data • Infrastructure • Structured capabilities/gap analysis against Requirements Week 0 Week 4 Week 13 Week 16 www.assessmentgroup.org

29. More on the requirements definition component • Key to a successful project is being able to manage the requirements development process & achieve stakeholder buy in • Experience, focus and a proven methodology for generating requirements is critical • Need to get requirements at the right level of detail/complexity • Need to be able to drive out ambiguity in the defined requirements • Define what the project is • Define what the project is not • Need to get all stakeholders aligned • Experience in leading major technology transformations is invaluable in managing such a project • Finally, all the above must be wrapped up into a carefully thought out plan with time, cost, technology and resource estimates • Doing this right typically takes 12-16 weeks of focused effort by a skilled, experienced team www.assessmentgroup.org

30. Key considerations for the Strategy / Planning phase • End-state objectives and expectations should be determined up front and all key stakeholders aligned • Weeks of doing this up front will save years in aiming at moving targets • Stakeholders almost always not aligned on even going-in assumptions • Propensity to “jump to answers” without the right information is a common source of failed projects • Technology is not “the answer” – it is an enabler • Business requirements and change management implications are the key pieces to get right - technology needs are by-products • Most frequent pitfall in these efforts is undefined and/or unclear business reqs. • Business requirements should be clearly defined and “approved” before technology discussions even begin • Getting the right answers in the shortest time and at the lowest cost requires leveraging different skillsets throughout • Strategic vs. Implementation • Education and business expertise vs. program management vs. technical www.assessmentgroup.org

31. ASG can bring key expertise to the Strategy / Planning phase • Requirements Definition • Collect and synthesize requirements from key stakeholders (structured interviews, research) • Define and support the process for aligning stakeholders • Translate Business Requirements into Technical Requirements • Technology Evaluation • Evaluate existing technologies and assessment platforms to gain an understanding of current capabilities, technologies, architectures, fit with the requirements • Surface and explore existing technology options/vendors to potentially leverage as a starting point to save time and costs • Example options: ITS, CAL, Vantage/McAnn, RANDA, Nimble Tools, AIR • Go Forward Plan Development • Develop structured plan for implementing the Next-Generation Platform • Roadmap that identifies key activities, priorities, and timing • Business case that identifies estimated costs/benefits and resources required www.assessmentgroup.org

32. Strategy / Planning Phase • Program and Vendor management • Oversight of vendor(s) selected to develop the platform per RFP specifications – e.g., project management, cost control and technical management • Maintain stakeholder involvement at the right points in the process • Future changes to the platform are carefully managed www.assessmentgroup.org

33. ASG can also provide support services for the “Design” phase • Functional and technical design and analysis • Develop Functional and Technical design / architecture based on the analysis of current platforms and guidance from the Consortia and advisory board • Basis for the Technology systems RFP • Current and future requirements • Open system/open source; Software as a Service/Applications store • Content and data management • Scalability and extensibility • Initial set of features and functionality/interoperability • Build vs. Buy analysis / recommendations • Provide recommendations based on business requirements, evaluation of the existing assessment platforms, and high level technical design • Explore different build alternatives (open source, university, vendor, other) and identify pros and cons for the Consortia • Findings/recommendations to be reflected in the RFP for vendor services www.assessmentgroup.org

34. ASG “Design” Support Services (continued) • RFP preparation and evaluation • Develop the RFP for the assessment technology system • Prepared and informed by lessons learned from previous evaluations • A variety of potential bidders to be solicited based on previous analyses • Evaluate/score vendor responses, both technical and cost, against a pre-determined rubric • Provide recommendations on the go-forward plan and vendor www.assessmentgroup.org

35. ASG “Implementation / Rollout” Support Services • Preparation of individual state financial models • The costs of converting to online assessment may be significant for many states. A comprehensive model and analysis detailing state-specific costs, benefits, breakeven points and return on investment in converting to OLA will help state departments of education and appropriate constituents understand the costs and issues in moving online • Systems Integration • Coordinate among different vendors/applications to ensure the development of a unified system • Investigate potential new technologies and applications • Develop specifications for applications interfaces • Manage the overall development of the system • Identify solutions for open source code management and administration www.assessmentgroup.org

36. ASG “Implementation / Rollout” Support Services (continued) • Rollout Plan Development • A comprehensive roll out plan for delivering the new assessment system should be developed well before actual completion of the system itself. Resource requirements, costs, time frames, a training plan and documentation along with a state by state implementation plan should be produced to ensure the rollout proceeds as smoothly as possible. • OLA Implementation “How To” Manual • Compilation of a detailed, step by step, approach to implementing online assessment including network, bandwidth and help desk requirements/activities • Lessons learned from previous state OLA implementations • Minimum system requirements • Minimum people requirements for implementation and ongoing support • Networks and bandwidth requirements www.assessmentgroup.org

37. Conclusions/Recommendations • Developing the next-generation assessment platform is a once in a generation opportunity and should result in a transformative change in the way assessment is thought about and delivered • To be successful, a significant and focused effort around planning, requirements, technology evaluation/design/ implementation and management of the effort is needed • ASG offers a sound approach for supporting the Consortia in successfully delivering the next-generation assessment platform • Core strategic and tactical technologists with deep expertise gained from within and outside of the Education industry • An advisory board consisting of representatives of brand name technology development companies • Skillful use of outside consultants offers a sound approach to successfully delivering the next generation assessment system www.assessmentgroup.org

38. Why ASG? • While the consortia are familiar with ASG’s cost modeling and technical assessment capabilities, the team assembled for this effort (Topol, Whitchurch & Co.) has built and/or overseen the building of many complex Technology projects • ASG understands the complexities of the consortia assessment systems as well as, assessment technology requirements • The group is independent and will bring an independent perspective and analysis of competing alternatives • The management approach to the effort minimizes overhead and maximizes value. Funding will go entirely to the project • The team and structured approach to managing the platform development effort will result in a powerful next generation assessment system www.assessmentgroup.org

39. Appendix – Advisory Board and Team Bios www.assessmentgroup.org

40. Advisory Board Bios – Charles Fadel • Global Education Lead at Cisco Systems • Cisco board member at the Partnership for 21st Century Skills (Cisco was a founding member), Innovate/Educate and Change the Equation (two organizations dedicated to advocacy of Science, Technology, Engineering and Math (STEM) • Vice-chair of the Education committee of the Business and Industry Advisory Committee (BIAC) to the Organization for Economic Co-operation and Development (OECD) and works with several teams at the OECD - PISA, AHELO, PIAAC, CELE, and TALIS • Worked with education ministries and organizations in more than thirty countries and states including Massachusetts, Canada (Federal, New Brunswick and PEI), France, Finland, Sweden, Italy, Chile, Brazil, Costa Rica, Tunisia, the Dominican Republic and others • Co-authored 21st Century Skills: Learning for Life in our Times (book) and frequently lectures on this topic, STEM, and Education Technology www.assessmentgroup.org

41. Advisory Board Bios – Charles Fadel (continued) • Work spans the continuum of Schools, Higher Education, and Workforce Development/Lifelong Learning • Visiting Practitioner at Harvard Graduate School of Education, exploring curriculum redesign concepts • Visiting Scholar at Wharton/UPenn where he recently taught a class on Technologies for Learning (receiving highest student rating) • Distinguished Guest to the President’s Council of Olin College • Incubating DigData, a non-profit addressing the convergence of 21st Century Skills and psychology to better teach Statistics and Probabilities • Creative Advisor to the Hayden planetarium of the Boston Museum of Science on their new implementations • Advising innovative school systems in Brazil (Lumiar) and Chile (Innova100) • Recently served on the Massachusetts Governor’s Readiness Project as well as its 21st Century Skills task force • Contributed many articles and has been interviewed in publications such as Technology & Learning, New Media Consortium, Massachusetts Association for Supervision and Curriculum Development, eSchool News, Education Week, University Business, EETimes, and others www.assessmentgroup.org

42. Advisory Board Bios – Charles Fadel (continued) • Spoke at numerous Education conferences, including the Consortium for School Networking (COSN), the National School Boards Association (NSBA), the National Center for Technology Innovation (NCTI), and the Masie Center’s learning conferences • Awarded five patents on video, content, and communication technologies • Holds a Bachelor of Science in electronics with course concentration in quantum and solid-state physics with a minor in neuroscience, and a Master of Business Administration in international marketing • An avid reader, he has auto-didactically learned disciplines such evolutionary psychology and comparative linguistics. He also enjoys the lessons of classical history www.assessmentgroup.org

43. Advisory Board Bios – Hans Van Delden • Partner, Vice President, Technology Practice at Booz & Company • 15 years in Management Consulting focused on the consumer products, education services, media and retail industries • Led 10+ strategy, operations and technology programs for several leading Education companies – e.g.,: • Technology Strategy / Architecture for an Online Assessment solution (Educational Assessment Provider) – included an evaluation of the technology providers in the educational assessment market • Architecture Review (functional and technical) of custom-developed Online Assessment Platform (High-stakes State Educational Assessment Provider) • Assessment, Strategy and Roadmap for the Online Assessment Solutions (Educational Services company) – included developing a technology strategy and investment roadmap for a newly acquired online assessment platform • Information Business Strategy (Educational Services company) - evaluated several potential new information-based businesses for our client • Latin American Growth Strategy (High-stakes Educational Assessment Company) • Operations Diagnostic for Puerto Rican division (High-stakes Educational Assessment Company) - included an end-to-end assessment of the operations to identify organizational, process and automation improvements www.assessmentgroup.org

44. Advisory Board Bios – Hans Van Delden (continued) • Experience working in 12 countries across North America, Europe and Asia • Twice awarded Booz & Company’s Professional Excellence Award, our highest award for client service • Earned five degrees from the Massachusetts Institute of Technology after five years of study • Holds an MBA and a Master of Science in Civil Engineering • Holds Bachelor of Science degrees in Mathematics, Civil Engineering and Management • MIT’s sole Rhodes Scholar candidate and Texas Rhodes Scholar finalist in 1995 • Recognized for achieving the highest score in North America on the first general actuary exam in 1993 www.assessmentgroup.org

45. Advisory Board Bios – Steve Katz • 20+ years as a driving force and thought leader in technology innovation and strategy • Held a wide range of management roles, advancing innovation with IBM, Apple, and Sony Pictures Entertainment • Also served as an executive strategy consultant to leading technology firms including Microsoft, Sony Corporation and SAP • Serial entrepreneur and founder of a number of pioneering ventures backed by Kleiner Perkins and other top VCs • Currently, Steve is founding partner in a strategy consulting boutique called O8 Partners, based in Silicon Valley, which helps drive major partnerships and business strategy for emerging technology companies • Currently, entrepreneur-in-residence at MHS Capital, an early stage venture firm funded by prominent Silicon Valley founders and executives • Involved in non-profit and philanthropic activities, including serving as a founding member of the Full Circle Fund.  The Full Circle Fund partnered with outstanding social innovators to explore different approaches to Educational reform • Holds BS an MS degrees in Computer Science from Columbia www.assessmentgroup.org

46. Advisory Board Bios -John Behrens • Director, Networking Academy Learning Systems Development, Cisco, Inc. • Responsible for the design and development of the Networking Academy’s global on-line instructional offerings that integrate interactive curricular and assessment technologies with hands on activities. These curriculum, assessment, and gaming tools serve more than 700,000 students per semester in 165 countries across more than 10 languages. Over 100 million assessments have been delivered to date through the Cisco Networking Academy. • Dr. Behrens’ team integrates complex simulations in instruction and assessment and developed authoring tools to enable instructor-led visualization of abstract networking concepts. • John’s research interests focus on the intersection of cognitive, statistical and computing issues in instruction, assessment, and on-line learning. Dr. Behrens has published chapters in a number of widely used psychology reference books along with articles in a range of educational, psychological, and statistical journals. • Adjunct Assistant Research Professor, Department of Psychology, University of Notre Dame (Indiana, USA). www.assessmentgroup.org

47. Advisory Board Bios -John Behrens (continued) • Master’s Degree in Special Education and Ph.D. in Educational Psychology, Arizona State University. • B.A. Psychology and Philosophy, University of Notre Dame. • Prior to joining Cisco, John was a tenured Associate Professor of Psychology in Education, Arizona State University. www.assessmentgroup.org

48. Advisory Board Bios – Barry Topol, CPA, MBA • CPA, MBA • Managing Partner, Assessment Solutions Group • Designed and co-developed ASG’s assessment cost model • Chief Financial Officer, Harcourt Assessment • Drove the review, analysis and ultimate restructuring/outsourcing of Harcourt Assessment’s online assessment platform • Established relationships with outside “best of breed” technology partners • Chief Financial Officer for several technology companies • Served as CIO/CFO for internet based start-up of a Fortune 500 Corporation; successfully developed the company’s website. Company won a “Webby” award for best online yellow pages site. • C-Level experience in educational publishing, high technology, internet and other industries • Managed technology departments and key technology initiatives in several companies www.assessmentgroup.org

49. Advisory Board Bios – Edward Roeber, Ph.D. • 40 years of experience in Assessment • Adjunct Professor of Education, Measurement and Quantitative Methods, College of Education, Michigan State University • Teaches courses on educational measurement • Works on projects to improve the assessment skills of prospective and current educators • Provides support for faculty and students on assessment • Senior Executive Director in the Michigan Department of Education OEAA • Responsible for overseeing the assessments of general education students, students with disabilities and ELLs, as well as the accreditation and accountability programs • Conducted statewide online assessment pilot in 2006 www.assessmentgroup.org

50. Advisory Board Bios – John F. Olson, Ph.D. • 25+ years of experience providing services, technical assistance, and support to states, the USED, CCSSO, testing companies, researchers, and others • Served as a technical advisor and independent consultant on a variety of measurement and statistical issues for international, national, state, and local assessment programs • Served in key leadership roles on several Enhanced Assessment Grants from the U.S. Department of Education. • Vice President for Psychometrics and Research Services, Harcourt Assessment, Inc. • Deep experience with online assessment including: • Conducting surveys of states on their use of CBT (for TILSA SCASS) • Authoring two published reports on the comparability of CBT-PPT • Serving as technical advisor to states implementing an OLA.  www.assessmentgroup.org