aps new team training session n.
Skip this Video
Loading SlideShow in 5 Seconds..
APS New Team Training Session PowerPoint Presentation
Download Presentation
APS New Team Training Session

Loading in 2 Seconds...

play fullscreen
1 / 59

APS New Team Training Session - PowerPoint PPT Presentation

  • Uploaded on

APS New Team Training Session. Introductions. Jeff Seaman. GEM Data Consultant Based in the US, Jeff joined GEM in 2007 to work on data quality issues.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'APS New Team Training Session' - macon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
jeff seaman
Jeff Seaman
  • GEM Data Consultant
  • Based in the US, Jeff joined GEM in 2007 to work on data quality issues.
  • Jeff holds an undergraduate and multiple graduate degrees from Cornell University and has worked extensively in the areas of technology, survey design, and sampling. He currently serves as the Co-Director of the Babson Survey Research Group. He has served as the Director of the Computing Resource Center and as Associate Vice Provost for Computing for the University of Pennsylvania and as Chief Information Officer for Lesley University.
yana litovsky
Yana Litovsky
  • GEM Data Manager
  • Based in the US, Yana Litovsky joined GEM in March, 2009. Her role as Data Manager involves commutating with National Teams about data related issues as well as collecting and processing the submitted APS data.
  • Before GEM, she spent 4 years at the Harvard Business School as a Research Associate working in the field of Industrial Psychology. She holds an Undergraduate degree from Brandeis University in Psychology and Philosophy.
niels bosma
Niels Bosma
  • GEM Research Fellow
  • Based in the Netherlands, Niels Bosma has been a member of the GEM Netherlands Team since 2001 and has been engaged in the GEM coordination team since 2004.
  • Niels Bosma holds an MSc degree in econometrics as well as a PhD on entrepreneurship dynamics and regional economic development at Utrecht University, Department of Economic Geography. He has co-authored the GEM Global Report since 2006, published a number of articles in academic journals and contributed to several books and research reports on entrepreneurship.
alicia coduras martinez
Alicia Coduras Martinez
  • GEM NES Coordinator
  • Based in Spain, Alicia Coduras heads the GEM Team in Spain since 2000 and manages the NES as a member of the GEM Coordination Team. She is also a member of the Research Committee.
  • Alicia is also an Associate Research Professor in the Department of Entrepreneurship and Family Business. She holds a doctorate in political sciences from the University Pompeu Fabra and a degree in economics and business sciences from the University of Barcelona. She publishes regularly in academic reviews and develops entrepreneurship reports for Spanish governmental institutions.
chris aylett
Chris Aylett
  • GEM Project Administrator
  • Based in the UK, Chris Aylett has been with GEM for over 4 years. He manages the paperwork from the National Teams, issues invoices, liaises with the accounts department and maintains and develops the GEM website. He is also the secretary to the Board, organizing their meetings and preparing agendas, papers and minutes.
  • Prior to working for GEM, he studied French at University College London, and is a qualified translator. Chris is also the bass guitarist for the up-and-coming rock band 'Scholars'.
what makes gem unique
What makes GEM unique?
  • not-for-profit academic research consortium, started in 1998,which estimates the participation in business start-ups and new firms in different countries.
  • grew from 10 countries in 1998 to 55 economies and over 160,000APS respondents in 2011 and about 86 economies overall.
  • comprised of Adult Population Survey (APS) and National Expert Survey (NES).
  • the Early Stage Entrepreneurial Activity prevalence rate (TEA index) is one of GEM’s best known measures of entrepreneurial activity.
gem objectives
GEM Objectives?
  • to measure differences in the level of entrepreneurial activity between countries -- because all national teams use the same procedure and provide data to the same GEM standards, the results provided by different countries can be compared worldwide.
  • to uncover factors determining the levels of entrepreneurial activity
  • to identify policies that may enhance entrepreneurship
  • to cover a greater proportion of OECD and non-OECD nations in the interests of gaining a detailed picture of the world's entrepreneurs and their role in economic development
  • to set international standards for social survey based research methodologies in entrepreneurship
some aps facts
Some APS Facts
  • The Adult Population Survey (APS) investigates the role of the individual in the lifecycle of the entrepreneurial process, and his or her characteristics or actions.
  • The APS is unique because it studies entrepreneurship through peoples’ motives and attitudes as they strive to set up, start and maintain businesses.
  • A representative sample of the adult population is used to measure the entrepreneurial activity that represents the entire country.
aps questionnaire1
APS Questionnaire
  • The APS is comprised of MODULES of different groups of questions
    • Core APS questions (mandatory)
    • Special Topics (2012) (mandatory)
      • Immigration
      • Business relationships
    • Innovation Questions (optional)
    • Network Question (optional)
  • The modules are distributed within the questionnaire in different BLOCKS for different target respondents.
aps blocks
APS Blocks
  • Introduction: All respondents
  • Block 5: Reserved
  • Block 6: Reserved
  • Block 7: DEMOGRAPHICS (All respondents)
aps block 1
APS Block 1

Q1A1 Q1A2



Don’t know






aps block 2
APS Block 2



Don’t know





Not Same Business


No/Same business



aps questionnaire2
APS Questionnaire
  • Full questionnaire, with codes and skip patterns.
sample design1
Sample Design
  • It is important to understand how the data you are analyzing has been collected.
  • Critical issues include:
    • Sample strata?
    • Any oversamples?
    • Different methodology?
  • Some national teams do not collect a sample that is proportional to the national population distribution – that add an oversample.
  • Required to meet the n=2000 minimum not counting any oversamples.
  • Any analysis needs to account for the inclusion (or not) of the oversample
differing methodology
Differing Methodology
  • Sample can be:
    • Face-to-face + fixed line
    • Mobile + fixed line
    • Face-to-face + mobile + fixed line
  • Each methodology can have its own bias
    • Age and gender distribution
    • Education, region, income, propensity to start a business
sample size
Sample Size
  • Minimum APS sample size = 2000
  • Sample sizes vary by nation.
  • Sample sizes vary by year for each nation.
rfp package
RFP Package
  • Request for Proposal (RFP) package is posted to the GEM consortium web site and an announcement is emailed to all teams in January or February of each year.
  • The RFP package includes all documents needed by National Team to submit APS/NES proposal, administer the questionnaires, record and submit all data. It also includes valuable reference information.

RFP refers to the entire document package as well as the RFP document itself.

rfp package aps questionnaire description
RFP Package(APS Questionnaire Description)

Detailed description of each question

rfp package spss data input template
RFP Package(SPSS Data Input Template)

Use provided SPSS Data Input Template to submit final survey data.

Data in any other format will not be processed

rfp package survey report methodology overview
RFP Package(Survey Report: Methodology Overview)

Provide full details of sample design (usually completed by vendor)

rfp package survey report sample strata fieldwork report
RFP Package(Survey Report: Sample Strata & Fieldwork Report)
  • List and describe sample strata (if not strata used, national info)
  • Complete Fieldwork Report for each strata after data is collected
preparing a proposal
Preparing a Proposal
  • Download and carefully read the RFP package
  • Work with Survey Vendor to decide on appropriate survey methodology and fill out all required RFP documents.
    • Vendors must use the GEM APS forms, they cannot submit their own forms.
    • Vendors must also submit a Survey Vendor Proposal, for which we provide no template.
  • Customize your questionnaire and SPSS Data Input Template
    • Decide which optional modules you will include (if any)
    • Decide if you will add additional questions
    • Establish you demographic variable answer categories
    • Translate the questionnaire if necessary
tips for translating aps questionnaire
Tips for Translating APS Questionnaire
  • Work with vendor on translations
    • National team has more knowledge on the purpose of the questions
    • Vendors have more knowledge on how translated questions will be received by the respondents
    • Use back-translations to check if the questions have been translated well
submitting a proposal
Submitting a Proposal
  • Email the completed forms to data@gemconsortium.org (refer to RFP for specific instructions on how to label and email proposal documents).
  • The GEM Data team will review submitted documents and proposed sampling methodology and send National Team a review of their proposal in a new worksheet added to Survey Report document (G. Proposal Review).
  • The national team MAY NOT BEGIN its survey until after their proposal has been officially approved (via email) by GEM Data Team.
review of aps proposal
Review of APS Proposal
  • Proposal document review: has all the required material been submitted?
    • Every mandatory chart applicable to your proposal must be completed
    • All information must be submitted in required format
  • APS Questionnaire review: do team-added questions interfere with core APS questionnaire?
    • Content and placement of any additional APS questions must be approved by Data Team
    • Verbal introduction of the survey to a potential respondent are reviewed.
  • Methodology review: does the sample and survey design meet GEM APS standards?
    • Is the sample size and design appropriate?
    • Is sample stratification appropriate?
    • Is the timing of survey administration appropriate?
    • Will the sampling method guarantee a representative sample?
    • Is a sufficient number of call-backs proposed?
    • Have all data issues from previous years been addressed?

If necessary, the national team will be requested to submit any missing information.

If necessary, the national team will be asked to clarify specifics of the sample design or surveying process.

If necessary, the national team will be requested to revise sampling and survey plan to meet GEM standards.

before aps data collection
Before APS Data Collection
  • Pilot Study: New team or teams with new vendor will be required to conduct pilot study with a minimum of 100 respondents and submit results before continuing APS Data Collection.
    • Results (submitted using APS Data Input Template)will be reviewed and tested for:
        • Proper skip pattern
        • Correct variable coding
        • Proper use of data templates
        • No excessive missing data.
    • Any problems will be quickly communicated, so corrective action can be taken early.
    • If there are no problems with pilot data, National Team can include pilot respondents in final sample.
    • A team must guarantee that any problem identified in the Pilot Data will be addressed and corrected during main sample data collection.
  • Mentor Team: New teams are recommended to find a find a mentor GEM country. Contact other GEM teams with same cultural background, especially if they have the same language.
during aps data collection
During APS Data Collection
  • Interim Data Submission: All teams should submit interim data file so that it can be checked for data format, skip logic or any other errors.
  • Data Quality Syntax: A national team can use SPSS syntax file provided with RFP package to check their data during data collection.
  • Whatever method is used to check data during data collection, ensure a way to remove or minimize the following data collection errors:
    • Skip logic errors due to incorrect questionnaire programing or poorly trained interviewers.
    • Unrepresentative sample: If final sample does not reflect the age/gender distribution of country within an acceptable range, team’s data may not be acceptable.
submitting aps dataset
Submitting APS Dataset
  • Submit APS Data in SPSS template. Datasets which do not meet GEM standards as indicated in the RFP will not be accepted.
  • Complete Strata & Fieldwork Report in Survey Report and submit with APS Data.
  • Send both documents to data@gemconsortium.org
initial review of aps data
Initial Review of APS Data
  • Initial data checks:
    • Data is correctly recorded and formatted
    • Unique IDs value for each respondent
    • Proper variable and value labels are used
    • All mandatory questions have been asked
  • Weights (if provided)
    • Representative of age/gender distribution in country and, if sample is stratified, if they are representative of strata distribution.
    • No excessive number of high value weights
  • Open-ended responses
    • Recorded in full and translated into English (if applicable)
    • Both native and English language responses provided in appropriate variables.
  • Skip logic checks
    • No excessive missing data
    • If data missing, how critical are the errors?
data quality report
Data Quality Report
  • Comprehensive review of data quality and team performance.
  • Included in new worksheet in Survey Report and sent to National Teams with or shortly after first results are available.
  • Includes:
    • % missing for key variables
    • % refused for key variables
    • % skip logic errors
    • Sample distribution
    • Deadlines
    • Proposal/APS Issues
    • Overview of APS weights
aps data processing levels
APS Data Processing Levels

1. Raw Data: Data submitted by National Team

2. Harmonized Data: Reviewed, cleaned, standardized data ready to be merged with all other countries for processing.

3. Processed Individual Level Data: APS Data with all computed APS indices (TEA rates, Established Business rates, etc.)

4. Processed National Level Data: APS index rates computed at the national level and weighted for 18-64 population. Released at 3 stages of completeness.

statistical help
  • Statistics Without Borders
    • Statistics Without Borders is an apolitical organization under the auspices of the American Statistical Association, comprised entirely of volunteers, that provides pro bono statistical consulting and assistance to organizations and government agencies.
  • Data Without Borders
    • Data Without Borders seeks to match non-profits in need of data analysis with freelance and pro bono data scientists who can work to help them with data collection, analysis, visualization, or decision support.
  • All analysis of GEM APS data MUST use weights.
  • Weights differ by the type of sample for each country.
  • Weights differ according to the age range for each country.
  • Weights do NOT account for different sample sizes.
vendor provided weights
Vendor-provided Weights
  • Describe how weights are calculated
  • All GEM APS weights have a mean of 1.0.
  • Vendor-supplied weights are checked – must match national age/gender distribution.
  • The design of your sample (use of strata) will dictate the weighting factors.
  • Strata-based weights must match overall strata sizes.
  • Very large values for weights (high leverage cases) are not allowed.
gem national aps weights
GEM National APS Weights
  • Weight_L: Used for the population aged 18 to 64 (L = Labor Force).
  • Weight_A: Used for the entire adult population aged 18 to 99 (A = Adults)
  • If national team only collected data for ages 18 to 64, then Weight_L and Weight_A are identical.
getting permission
Getting Permission
  • During year of data collection: National team access to own data only.
  • During first year after Global Report press release: National team access to own data only.
  • Can get access with specific permission of each country for use of their data.

March 14


April 30

APS RFP (Methodology Overview Form and Survey Vendor Proposal) due

July 16

APS & NES data and documentation, including Fieldwork Report due

Week of September 17

First release of APS & NES master dataset on website

Week of October 13

Second release of APS master dataset uploaded to website

End of November

Final release of APS master dataset uploaded to website

crtical lessons