1 / 68

APS TRAINING SESSION

APS TRAINING SESSION. Yana Litovsky Jonathan Francis Carmona. This session will cover:. APS tools & guidelines APS Questionnaire Survey Design Weights Sample Size Step in APS Cycle: Proposal Submission Data Collection Data Review Processed Results.

herbst
Download Presentation

APS TRAINING SESSION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. APS TRAINING SESSION Yana Litovsky Jonathan Francis Carmona

  2. This session will cover: APS tools & guidelines • APS Questionnaire • Survey Design • Weights • Sample Size Step in APS Cycle: • Proposal Submission • Data Collection • Data Review • Processed Results

  3. the Adult Population Survey (APS) investigates the role of the individual in the lifecycle of the entrepreneurial process and his or her characteristics or actions • a representative sample of the adult population is used to estimate the entrepreneurial activity that represents the entire country. • The APS is unique because it measures • Motives • Attitudes • Actions • Business characteristics Some APS Facts

  4. GEM Entrepreneurial Phases

  5. APS Questionnaire

  6. The APS is comprised of MODULES of different groups of questions, often on different topics • The modules are distributed within the questionnaire in different BLOCKSfor different target respondents APS Questionnaire

  7. Introduction: All respondents Block 1: Nascent entrepreneurs Block 2: Owner-managers Block 3: Potential and discontinuing Block 4: Informal investors Block 5: All respondents (occupation) Block 6: Reserved (not used in 2014) Block 7: All respondents (demographics) APS Blocks

  8. Core APS Module • Remain consistent year to year • Used to derive main GEM indicators • Must be included by all GEM National Teams • Special Topic Module • Set of questions that change year to year • On different research topics • May be included in one or more blocks • May include optional questions • Optional Modules • Questions on different topics • Suggested by national teams or derived from previous Special Topics • Follow placement instructions APS Modules

  9. Block 1: Nascent entrepreneurs (flow) INTRODUCTION REQUIRED QUESTIONS FOR ENTIRE SAMPLE OPTIONAL QUESTIONS FOR ENTIRE SAMPLE FILTER QUESTIONS FOR ALL RESPONDENTS BLOCK1 QUALIFIED RESPONDENTS REQUIRED QUESTIONS FOR NASCENT ENTREPRENEURS SPECIAL TOPIC: REQUIRED QUESTIONS FOR NASCENT ENTREPRENEURS UN-QUALIFIED RESPONDENTS OPTIONAL QUESTIONS FOR NASCENT ENTREPRENEURS BLOCK2

  10. Block 2: Owner-managers (flow) FILTER QUESTIONS FOR ALL RESPONDENTS BLOCK2 QUALIFIED RESPONDENTS REQUIRED QUESTIONS FOR OWNER MANAGERS SPECIAL TOPIC: REQUIRED QUESTIONS FOR OWNER MANAGERS UN-QUALIFIED RESPONDENTS OPTIONAL QUESTIONS FOR OWNER MANAGERS BLOCK3

  11. Blocks 3 & 4 FILTER QUESTIONS FOR ALL RESPONDENTS BLOCK3 REQUIRED QUESTIONS FOR POTENTIAL & DISCONTINUING QUALIFIED RESPONDENTS OPTIONAL QUESTIONS FOR POTENTIAL & DISCONTINUING UN-QUALIFIED RESPONDENTS BLOCK4 REQUIRED QUESTIONS FOR INFORMAL INVESTORS OPTIONAL QUESTIONS FOR INFORMAL INVESTORS

  12. Blocks 5 & 7 BLOCK5 REQUIRED QUESTIONS FOR ALL (OCCUPATION) SPECIAL TOPIC: REQUIRED QUESTIONS FOR ALL (OR BASED ON OCCUPATION STATUS) OPTIONAL QUESTIONS FOR ALL (OR BASED ON OCCUPATION STATUS) BLOCK7 REQUIRED QUESTIONS FOR ALL (DEMOGRAPHICS) REQUIRED VENDOR PROVIDED INFORMATOIN OPTIONAL QUESTIONS FOR ALL

  13. Full questionnaire, with codes and skip patterns. APS Questionnaire Format

  14. Each block is color-coded APS Questionnaire Format

  15. Survey Design

  16. There is no one standard GEM methodology! • GEM’s only fixed requirement is that a team collects a nationally representative sample of at least 2000 adults • To ensure this, GEM provides guidelines about: • Proper choice of respondent contact method • Sufficient number of callbacks and initial contact attempts • Appropriate time of survey administration • Appropriate stratification of sample Some guidelines stable but others differ from team to team Survey Design Guidelines

  17. Sample can be using any combination of: • Face-to-face • Fixed line • Mobile phone • Choice based on feasibility, budget & bias • Each methodology can have its own bias, for example: • Fixed-line usually biased toward older respondents • Mobile phone usually biased toward younger respondents • Face-to-face may be biased against people often out of the house Contact Method Options

  18. Most teams collect a sample that is proportional to the national population distribution Others may add an oversample: may collect disproportionately more responses in a metropolitan area Required to meet the n=2000 minimum not counting any oversamples. Oversample may be part of Global Report sample or submitted separately Sample Design Options

  19. Weights

  20. All analysis of GEM APS data MUST use weights. National weight mean =1 (do not account for different sample sizes). Weights differ by the type and age/range of sample. Weighted age/gender distribution must be nationally representative. Strata-based weights must match overall strata sizes. Very large values for weights (high leverage cases) are not allowed. Weights

  21. RESPONSE WEIGHTS: Used to achieve a better match of the sample surveyed to that of the overall population. • To reduce potential bias due to different responses rates or normal sampling variability. • Required for every country. • SAMPLE DESIGN WEIGHTS: Used to adjust a sample that was purposefully designed to have unequal representation • For example, if a team oversamples (by design) in a metropolitan area, weights are used to not over-represent the city respondents. Types of Weights

  22. WEIGHT_L • Used for the adult population aged 18 to 64 (L = Labor Force). • WEIGHT_A • Used for the entire adult population aged 18 to 99 (A = Adults) • If national team only collected data for ages 18 to 64, then Weight_L and Weight_A are identical. GEM National APS Weights

  23. Only provided for teams who request regional-level analysis. • WEIGHT_LR • WEIGHT_AR • Each region has its own weights calculated as if they were the entire nation. • Should be used ONLY for region-to-region comparisons, not for any national-level analysis! GEM Regional APS Weights

  24. Always use weights for any GEM APS individual-level analysis. If you want to find comparable Global Report results, use WEIGHT_L Final Lesson

  25. Sample Size

  26. Sample Sizes (2013) Minimum sample N = 2000 Appropriate regardless of national population size

  27. respondents per block (fraction of sample)

  28. respondents per block(number in 2000 person samples)

  29. But sample sizes vary by nation and year. Beware when combining data from nations or years with different sample sizes. Weights do NOT adjust for different sized samples The weighted total for any national analysis will match the unweighted total (that is why weights have a mean of 1). Sample Size:combining individual level data

  30. Proposal Submission Process

  31. Request for Proposal (RFP) is posted to the GEM consortium website and an announcement is emailed to all teams in February of each year. The RFP includes all documents needed by National Team to submit APS/NES proposal, administer the questionnaires, record and submit all data. It also includes valuable reference information. “RFP” refers to the entire document package as well as the Request for Proposal document itself. RFP Package

  32. Data Delivery Documents: • SPSS Data Input Templates (APS & NES) • Proposal Documents: • Instructions for submitting proposals (APS & NES) • Survey Reports (APS & NES) • Additional NES instructional documents • Questionnaire Documents: • Final version of questionnaires (APS & NES; English & Spanish) • APS question-by-question commentary • Reference Documents: • PowerPoint Training presentation (APS & NES) • GEM Timeline • Reference Information (additional APS questions, education categories, optional module) RFP Package (contents)

  33. Use provided SPSS Data Input Template to submit final survey data. Data in any other format will not be processed RFP PackageData Delivery: SPSS data input file

  34. Detailed description of each question RFP PackageQuestionnaire: APS commentary

  35. ~ Excel document comprised of several worksheets ~ Unique for each team • Methodology Overview • Previous year’s Data Quality Report • APS Questionnaire information • Sample strata description & fieldwork report • Population statistics • Weight construction • Education categories • Additional worksheets populated during APS cycle RFP PackageProposal: APS Survey Report

  36. Worksheet A: Methodology Overview • Provide full details of sample design (usually completed by vendor) • Information included in Global Report RFP PackageProposal: APS Survey Report • Sample size • Interview method • Sampling methodology • Sample stratification • Call backs • Previous data issues

  37. Worksheet C: Strata & Fieldwork Report • List and describe sample strata (if none used, national information) • Complete fieldwork report for each strata after data is collected RFP PackageProposal: APS Survey Report

  38. Download and carefully read the RFP package • Work with Survey Vendor to decide on appropriate survey methodology and fill out all required RFP documents. • Vendors must use GEM forms except for Vendor Proposal(has no template) • Customize your questionnaire and SPSS Data Input Template • Decide which optional modules you will include (if any) • Decide if you will add additional questions (must be approved by GEM) • Establish you demographic variable answer categories • Translate the questionnaire if necessary Final questionnaire is each team’s responsibility Preparing an APS Proposal

  39. Work with vendor on translations • National team has more knowledge of question purpose • Vendor can advise how questions will be received by respondent • Use question-by-question commentary • Always do a back-translation • Can share other team’s translations with permission Tips for translating APS questionnaire

  40. ~ Team creates own categories for education variable ~ GEM recodes categories into harmonized variables Keep our recoding rules in mind when constructing your education variable • Respondent must complete an education level (e.g. earn a degree) to be coded in that category • Bachelors, Masters and Doctorate degrees are best kept in separate categories Tips for EDUCATION categories

  41. Team creates their own categories for income variable • GEM requires at least 6 income ranges • GEM recodes variable into lower, middle, upper percentile Keep national income distribution in mind! If most respondents fall into one income range, this variable is not useful. Tips for INCOME categories

  42. Email the completed forms to data@gemconsortium.org (refer to RFP for instructions on how to label documents). • The GEM Data team will review submitted documents and proposed sampling methodology and send National Team a review in a new worksheet added to Survey Report document (G. Proposal Review). • The national team MAY NOT BEGIN APS survey until their proposal has been officially approved (via email) by GEM Data Team. Submitting an APS Proposal

  43. Proposal document review: has all the required material been submitted? • Every mandatory chart applicable to your proposal must be completed • All information must be submitted in required format • APS Questionnaire review: do teamquestions interfere with core APS? • Content and placement of any additional APS questions must be approved by Data Team • Verbal introduction of the survey to a potential respondent are reviewed(see RFP tips) • Methodology review: does sample/survey design meet GEM standards? • Is sample stratification appropriate? • Is the timing of survey administration appropriate? • Is a sufficient number of call-backs and initial contact attempts proposed? • Have all data issues from previous years been addressed? Evaluation of APS Proposal

  44. If necessary, the National Team will be requested to: submit any missing information clarify specifics of the sample design revise sampling and survey plan Evaluation of APS Proposal

  45. DO NOT WAIT FOR PROPOSAL TO BE COMPLETE BEFORE GETTING IN TOUCH WITH GEM DATA TEAM IF YOU HAVE QUESTIONS ~ We can give suggestions about choice of vendor ~ Pre-evaluate different methodologies ~ Help team conduct a pilot test

  46. Data Collection Process

  47. Mentor Team: New teams are recommended to find a find a mentor GEM country. Contact other GEM teams with same cultural background, especially if they have the same language. Before APS Data Collection

  48. Pilot Study: New team or teams with new vendor will be required to conduct pilot study with a minimum of 100 respondents and submit results before continuing data collection. • Results (submitted using APS Data Input Template) will be tested for: • proper skip pattern • correct variable coding • proper use of data templates • no excessive missing data • Any problems will be quickly communicated • If no problems, pilot respondents can be included in final sample. • A team must guarantee that any will be addressed and corrected Before APS Data Collection

  49. Interim Data Submission: If Pilot Study is not required, teams are still encouraged to submit interim data file to be checked for data format, skip logic or other errors. Data Quality Syntax: A national team can use SPSS syntax to check their data before submitting to GEM. These files replicate most GEM data checks. During APS Data Collection: How to check your data

  50. Skip logic errors due to incorrect questionnaire programing or poorly trained interviewers: If these result in excessive missing data, team may be asked to recontact participants (if possible) or resample, in order for the data to be approved and included in the Global Report (extra data is also a problem) During APS Data Collection: Errors to check and avoid

More Related