Redesigning French Internet Business data collection at Insee: the Coltrane project

1. Redesigning French Internet Business data collection at Insee: the Coltrane project 4th International Workshop on Internet survey methods (Daejeon)

2. Outline of the presentation • Thanks and context • The existing system in France • Surveys • Tools • The Coltrane project (four work packages) • Authentication • Contacts management • Portal of customized services • Data collection platform 4th International Workshop on Internet survey methods

3. 1.1 Business surveys and National Statistical Authorities (NSA) • NSA : Insee and 5 or 6 ministerial statistical Offices (MSO) (In 2010 Sessi, the MSO for industry, & Insee have merged) • 80 Business surveys (of many kinds) • 30 Insee and 50 MSO (+ others) • About 500,000 enterprises (in over 3,5 millions) receive at least one questionnaire • The sum of all the samples sizes is  700,000 • Taking into account the periodicity of the surveys (12 questionnaires for a monthly survey), 1 800,000 forms are sent each year by the NSA. • "sent" can be • By mail for paper forms • Through a website for dematerialized forms 4th International Workshop on Internet survey methods

4. 1.1 The present situation regarding Internet Usage • So far, put surveys on the web has not been a priority • 2012 situation: 4th International Workshop on Internet survey methods

5. 1.1 Factors explaning the ratio of Internet answers • Heterogeneous from one office to another • Heterogeneous from one survey to another • Possible factors • The age of the process • The type of variables collected • The number of variables and the length of the questionnaire • The frequency of the survey • The size of the responding firms • The renewal of the samples • The ergonomics of the human interface • the efforts made by Insee staff • ... 4th International Workshop on Internet survey methods

6. 1.2 The tools to implement Internet Surveys • First Internet survey : 2000 (Sessi) • Presently a second-generation application (ASP) • Now Insee took over the portal (since 2010) • Beginning at Insee : 2004 (application called CRPI) • This application is now 8 years old (JAVA) • Designed for repetitive short term surveys • For specific purposes, new ways of collecting dematerialized data have been developed recently: • Upload / download application (eg EXCEL sheets)(2009) • Blaise IS (Dutch software) (2009) • Voozanoo (a free PHP software from Epiconcept) (2008) • Conclusion: many tools ! Complex for us and for enterprises (e.g. one portal for each tool) 4th International Workshop on Internet survey methods

7. 1.2 An analysis grid to compare the tools (2010) • Originally based on the GSBPM • Within the sub-processes we have defined  23 functions such as : • Build data collection instrument (create the form) -  Create a form on one web page………………………… -  Create a form on several web pages or screens …….. -  Create simple filters (from one question to another)…. -  Create complex filters (from page to page)……………. • Build or enhance process components -   Generate both login & password………………………. -   Take into account the campaign dates ……………….. -   Customize the questionnaire (e.g. with previous answers of the same enterprise) • This grid helped us to analyse the pros & the cons of the tools and of the whole system 4th International Workshop on Internet survey methods

8. 1.2 There is a need for a new project • CRPI is the best; but • old (9 years) • based on a proprietary software (to implement java) • its maintenance is costly; • Not designed for big annual surveys • No common directory nor common governance nor common portal • Technically complex • moreover: in april 2011, the government requested that all statistical surveys should be put on the web; existing tools couldn’t allow it (specially for the SBS survey) 4th International Workshop on Internet survey methods

9. 2 The Coltrane project • Platform offering a range of technical & business services • Four independent work packages: • Authentication • Contact management • Portal of customized services (for contacts) • Data collection platform (divided into 4 blocks) • Each WP is composed of several functions accessible through services • A specific batch is planned to transform the SBS-survey into a web-based survey • Innovative because forms should be generated from their DDI description 4th International Workshop on Internet survey methods

10. 2.1 The Coltrane project: authentication • Independent from the collection itself • We do not think about using PKI(Public Key Infrastructure) but a LDAP directory to authenticate all the contacts • Any « contact » in an enterprise should have a unique login & password • We shall offer the possibility of using this service to all the MSOs • Enterprises do not make any difference whether the survey is carried out by Insee or a MSO • They receive questionnaires from both • But each ministry has its own Internet policy and have already built something for their own web-sites  we are not sure to be successfull ! 4th International Workshop on Internet survey methods

11. 2.2 The Coltrane project: contact management • Independent from the collection itself, but linked to the authentication service (to exchange credentials) • Can be used to manage all kinds of contacts (for example, people buying data) • Insee staff will be able to: • Create or delete a contact and its respective characteristics • Renew credentials • Display the list of contacts of a given enterprise • Display the list of the surveys of a given contact • Assist the web respondents and inform them about the coming surveys • Communicate with all the contacts of a given survey • ….. 4th International Workshop on Internet survey methods

12. 2.3 The Coltrane project: portal of customized services • A unique Internet portal will present all the surveys (different from today) • Including the MSO surveys (as far as they have accepted to use our portal) • The portal will be customized for each contact so that, once authenticated, he can: • only see the surveys he is interested in • modify his own personal parameters (phone number, e-mail, address, etc.) • transfer his “answering power” to a colleague • display his previous answers • look at the remaining surveys he still has to answer as well as their deadlines 4th International Workshop on Internet survey methods

13. 2.4 The Coltrane project: Data collection platform • Main part of the project • Block 1: offering different collection modes • Classical web form • Paper form (exchanged by mail) • Electronic data Interchange (EDI) • Upload / download service If several collection modes exist, the questionnaires should be generated from the same metadata flow • Block 2: running the collection (eg building management tools) • Opening/closing a campaign • Edit statistics for the collection • Organize reminders • Start litigations with non respondents 4th International Workshop on Internet survey methods

14. 2.4 The Coltrane project: Data collection platform • Block 3: generating questionnaires • Aims at maintaining the coherence between a questionnaire & its description (metadata) • We will use a DDI model to model the questionnaire • Then the form should be generated from its DDI description • We are testing some softwares helping either the generation or the DDI description • This could allow us to put the SBS survey on the web • Block 4: managing the collected data • In our view, centralizing all the collected data (whatever the data collection mode) is of utmost importance • The following functions will be developed : retrieve and control the data, adapt the formats of data flows, keep an image of the enterprise answer 4th International Workshop on Internet survey methods

15. 2.5 The Coltrane project: Some specific issues • Give the possibility for one enterprise to keep a copy of its answer and for several respondents in the same firm to fill up a single form • Find the best trade-off between • Sending back quickly new credentials to a contact (when he has lost them) • Keeping a good level of security • Use the dialogue possibilities of Internet • to help firms to answer to questions with hundreds of possible codes • to implement dynamic controls (eg with previous answers)or explanations (in case of “peculiar” answers) • Make a proof an enterprise has received a web form and has not replied (for mandatory surveys, to impose fines) • Legal and technical issue 4th International Workshop on Internet survey methods

16. Insee 18 bd Adolphe-Pinard 75675 Paris Cedex 14 www.insee.fr Informations statistiques : www.insee.fr / Contacter l’Insee 09 72 72 4000 (coût d’un appel local) du lundi au vendredi de 9h00 à 17h00 The Coltrane project is still in a conceptual phase and should be completed by 2014 or 2015. Thanks for your attention ! Contact M. Jean-Marc Béguin Tél. : +33 1 41 17 50 41 Courriel : jean-marc.beguin@insee.fr