1 / 59

APS New Team Training Session

APS New Team Training Session. Introductions. Jeff Seaman. GEM Data Consultant Based in the US, Jeff joined GEM in 2007 to work on data quality issues.

macon
Download Presentation

APS New Team Training Session

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. APS New Team Training Session

  2. Introductions

  3. Jeff Seaman • GEM Data Consultant • Based in the US, Jeff joined GEM in 2007 to work on data quality issues. • Jeff holds an undergraduate and multiple graduate degrees from Cornell University and has worked extensively in the areas of technology, survey design, and sampling. He currently serves as the Co-Director of the Babson Survey Research Group. He has served as the Director of the Computing Resource Center and as Associate Vice Provost for Computing for the University of Pennsylvania and as Chief Information Officer for Lesley University.

  4. Yana Litovsky • GEM Data Manager • Based in the US, Yana Litovsky joined GEM in March, 2009. Her role as Data Manager involves commutating with National Teams about data related issues as well as collecting and processing the submitted APS data. • Before GEM, she spent 4 years at the Harvard Business School as a Research Associate working in the field of Industrial Psychology. She holds an Undergraduate degree from Brandeis University in Psychology and Philosophy.

  5. Niels Bosma • GEM Research Fellow • Based in the Netherlands, Niels Bosma has been a member of the GEM Netherlands Team since 2001 and has been engaged in the GEM coordination team since 2004. • Niels Bosma holds an MSc degree in econometrics as well as a PhD on entrepreneurship dynamics and regional economic development at Utrecht University, Department of Economic Geography. He has co-authored the GEM Global Report since 2006, published a number of articles in academic journals and contributed to several books and research reports on entrepreneurship.

  6. Alicia Coduras Martinez • GEM NES Coordinator • Based in Spain, Alicia Coduras heads the GEM Team in Spain since 2000 and manages the NES as a member of the GEM Coordination Team. She is also a member of the Research Committee. • Alicia is also an Associate Research Professor in the Department of Entrepreneurship and Family Business. She holds a doctorate in political sciences from the University Pompeu Fabra and a degree in economics and business sciences from the University of Barcelona. She publishes regularly in academic reviews and develops entrepreneurship reports for Spanish governmental institutions.

  7. Chris Aylett • GEM Project Administrator • Based in the UK, Chris Aylett has been with GEM for over 4 years. He manages the paperwork from the National Teams, issues invoices, liaises with the accounts department and maintains and develops the GEM website. He is also the secretary to the Board, organizing their meetings and preparing agendas, papers and minutes. • Prior to working for GEM, he studied French at University College London, and is a qualified translator. Chris is also the bass guitarist for the up-and-coming rock band 'Scholars'.

  8. Brief GEM Overview

  9. What makes GEM unique? • not-for-profit academic research consortium, started in 1998,which estimates the participation in business start-ups and new firms in different countries. • grew from 10 countries in 1998 to 55 economies and over 160,000APS respondents in 2011 and about 86 economies overall. • comprised of Adult Population Survey (APS) and National Expert Survey (NES). • the Early Stage Entrepreneurial Activity prevalence rate (TEA index) is one of GEM’s best known measures of entrepreneurial activity.

  10. GEM Objectives? • to measure differences in the level of entrepreneurial activity between countries -- because all national teams use the same procedure and provide data to the same GEM standards, the results provided by different countries can be compared worldwide. • to uncover factors determining the levels of entrepreneurial activity • to identify policies that may enhance entrepreneurship • to cover a greater proportion of OECD and non-OECD nations in the interests of gaining a detailed picture of the world's entrepreneurs and their role in economic development • to set international standards for social survey based research methodologies in entrepreneurship

  11. Some APS Facts • The Adult Population Survey (APS) investigates the role of the individual in the lifecycle of the entrepreneurial process, and his or her characteristics or actions. • The APS is unique because it studies entrepreneurship through peoples’ motives and attitudes as they strive to set up, start and maintain businesses. • A representative sample of the adult population is used to measure the entrepreneurial activity that represents the entire country.

  12. Entrepreneurial Phases Identified in GEM

  13. APS Questionnaire

  14. APS Questionnaire • The APS is comprised of MODULES of different groups of questions • Core APS questions (mandatory) • Special Topics (2012) (mandatory) • Immigration • Business relationships • Innovation Questions (optional) • Network Question (optional) • The modules are distributed within the questionnaire in different BLOCKS for different target respondents.

  15. APS Blocks • Introduction: All respondents • Block 1: NASCENT-ENTRENREURS • Block 2: OWNER-MANAGERS • Block 3: POTENTIAL AND DISCONTINUING • Block 4: INFORMAL INVESTORS • Block 5: Reserved • Block 6: Reserved • Block 7: DEMOGRAPHICS (All respondents)

  16. APS Block 1 Q1A1 Q1A2 BLOCK1 Yes Don’t know NASCENT-ENTRENREURS (CORE) INTER FIRM NETWORKS(SPECIAL TOPIC) No NETWORK (OPTIONAL) TEAM-ADDED (OPTIONAL)

  17. APS Block 2 Q2A Yes Don’t know BLOCK2 + OWNER-MANAGERS (CORE) Q2C Not Same Business INTER FIRM NETWORKS (SPECIAL TOPIC) No/Same business NETWORK (OPTIONAL) TEAM-ADDED (OPTIONAL)

  18. APS Questionnaire • Full questionnaire, with codes and skip patterns.

  19. Sample Design

  20. Sample Design • It is important to understand how the data you are analyzing has been collected. • Critical issues include: • Sample strata? • Any oversamples? • Different methodology?

  21. Oversamples • Some national teams do not collect a sample that is proportional to the national population distribution – that add an oversample. • Required to meet the n=2000 minimum not counting any oversamples. • Any analysis needs to account for the inclusion (or not) of the oversample

  22. Differing Methodology • Sample can be: • Face-to-face + fixed line • Mobile + fixed line • Face-to-face + mobile + fixed line • Each methodology can have its own bias • Age and gender distribution • Education, region, income, propensity to start a business

  23. Sample Size • Minimum APS sample size = 2000 • Sample sizes vary by nation. • Sample sizes vary by year for each nation.

  24. Sample Sizes (2011)

  25. Fraction of Sample

  26. Number in N=2000 Sample

  27. Proposal Submission Process

  28. RFP Package • Request for Proposal (RFP) package is posted to the GEM consortium web site and an announcement is emailed to all teams in January or February of each year. • The RFP package includes all documents needed by National Team to submit APS/NES proposal, administer the questionnaires, record and submit all data. It also includes valuable reference information. RFP refers to the entire document package as well as the RFP document itself.

  29. RFP Package(APS Questionnaire Description) Detailed description of each question

  30. RFP Package(SPSS Data Input Template) Use provided SPSS Data Input Template to submit final survey data. Data in any other format will not be processed

  31. RFP Package(Survey Report: Methodology Overview) Provide full details of sample design (usually completed by vendor)

  32. RFP Package(Survey Report: Sample Strata & Fieldwork Report) • List and describe sample strata (if not strata used, national info) • Complete Fieldwork Report for each strata after data is collected

  33. Preparing a Proposal • Download and carefully read the RFP package • Work with Survey Vendor to decide on appropriate survey methodology and fill out all required RFP documents. • Vendors must use the GEM APS forms, they cannot submit their own forms. • Vendors must also submit a Survey Vendor Proposal, for which we provide no template. • Customize your questionnaire and SPSS Data Input Template • Decide which optional modules you will include (if any) • Decide if you will add additional questions • Establish you demographic variable answer categories • Translate the questionnaire if necessary

  34. Tips for Translating APS Questionnaire • Work with vendor on translations • National team has more knowledge on the purpose of the questions • Vendors have more knowledge on how translated questions will be received by the respondents • Use back-translations to check if the questions have been translated well

  35. Submitting a Proposal • Email the completed forms to data@gemconsortium.org (refer to RFP for specific instructions on how to label and email proposal documents). • The GEM Data team will review submitted documents and proposed sampling methodology and send National Team a review of their proposal in a new worksheet added to Survey Report document (G. Proposal Review). • The national team MAY NOT BEGIN its survey until after their proposal has been officially approved (via email) by GEM Data Team.

  36. Review of APS Proposal • Proposal document review: has all the required material been submitted? • Every mandatory chart applicable to your proposal must be completed • All information must be submitted in required format • APS Questionnaire review: do team-added questions interfere with core APS questionnaire? • Content and placement of any additional APS questions must be approved by Data Team • Verbal introduction of the survey to a potential respondent are reviewed. • Methodology review: does the sample and survey design meet GEM APS standards? • Is the sample size and design appropriate? • Is sample stratification appropriate? • Is the timing of survey administration appropriate? • Will the sampling method guarantee a representative sample? • Is a sufficient number of call-backs proposed? • Have all data issues from previous years been addressed? If necessary, the national team will be requested to submit any missing information. If necessary, the national team will be asked to clarify specifics of the sample design or surveying process. If necessary, the national team will be requested to revise sampling and survey plan to meet GEM standards.

  37. APS Data Collection Process

  38. Before APS Data Collection • Pilot Study: New team or teams with new vendor will be required to conduct pilot study with a minimum of 100 respondents and submit results before continuing APS Data Collection. • Results (submitted using APS Data Input Template)will be reviewed and tested for: • Proper skip pattern • Correct variable coding • Proper use of data templates • No excessive missing data. • Any problems will be quickly communicated, so corrective action can be taken early. • If there are no problems with pilot data, National Team can include pilot respondents in final sample. • A team must guarantee that any problem identified in the Pilot Data will be addressed and corrected during main sample data collection. • Mentor Team: New teams are recommended to find a find a mentor GEM country. Contact other GEM teams with same cultural background, especially if they have the same language.

  39. During APS Data Collection • Interim Data Submission: All teams should submit interim data file so that it can be checked for data format, skip logic or any other errors. • Data Quality Syntax: A national team can use SPSS syntax file provided with RFP package to check their data during data collection. • Whatever method is used to check data during data collection, ensure a way to remove or minimize the following data collection errors: • Skip logic errors due to incorrect questionnaire programing or poorly trained interviewers. • Unrepresentative sample: If final sample does not reflect the age/gender distribution of country within an acceptable range, team’s data may not be acceptable.

  40. Submitting APS Dataset • Submit APS Data in SPSS template. Datasets which do not meet GEM standards as indicated in the RFP will not be accepted. • Complete Strata & Fieldwork Report in Survey Report and submit with APS Data. • Send both documents to data@gemconsortium.org

  41. APS Data Review Process

  42. Initial Review of APS Data • Initial data checks: • Data is correctly recorded and formatted • Unique IDs value for each respondent • Proper variable and value labels are used • All mandatory questions have been asked • Weights (if provided) • Representative of age/gender distribution in country and, if sample is stratified, if they are representative of strata distribution. • No excessive number of high value weights • Open-ended responses • Recorded in full and translated into English (if applicable) • Both native and English language responses provided in appropriate variables. • Skip logic checks • No excessive missing data • If data missing, how critical are the errors?

  43. Data Quality Report • Comprehensive review of data quality and team performance. • Included in new worksheet in Survey Report and sent to National Teams with or shortly after first results are available. • Includes: • % missing for key variables • % refused for key variables • % skip logic errors • Sample distribution • Deadlines • Proposal/APS Issues • Overview of APS weights

  44. Data Quality Report

  45. Data Quality Report

  46. APS Processed Results

  47. APS Data Processing Levels 1. Raw Data: Data submitted by National Team 2. Harmonized Data: Reviewed, cleaned, standardized data ready to be merged with all other countries for processing. 3. Processed Individual Level Data: APS Data with all computed APS indices (TEA rates, Established Business rates, etc.) 4. Processed National Level Data: APS index rates computed at the national level and weighted for 18-64 population. Released at 3 stages of completeness.

  48. TEA creation flowchart

  49. StatisticalHelp • Statistics Without Borders • Statistics Without Borders is an apolitical organization under the auspices of the American Statistical Association, comprised entirely of volunteers, that provides pro bono statistical consulting and assistance to organizations and government agencies. • Data Without Borders • Data Without Borders seeks to match non-profits in need of data analysis with freelance and pro bono data scientists who can work to help them with data collection, analysis, visualization, or decision support.

  50. Weights

More Related