1 / 45

UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

Results of the questionnaire on Census Data Processing ( A total of 9 questionnaires were received). UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008. Fathia ABDELFADIL UN-ESCWA.

blake-hunt
Download Presentation

UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Results of the questionnaireon Census Data Processing(A total of 9 questionnaires were received) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008 Fathia ABDELFADIL UN-ESCWA UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  2. Part B:DATA CAPTURE • Question 1: Does your office use any of the following methods for census/survey data capture? • Manual data entry • Optical Mark Recognition (OMR) • Optical Character Recognition/Intelligent Character Recognition (OCR/ICR) • Personal Digital Assistant (PDA) • Internet • Other UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  3. 9 countries replied. - 2 countries : Manual data entry & other ( -Syria: manual using CSPro - Iraq: manual + traditional system analysis & programming ) - 3 countries : Manual data entry and PDA (- Jordan: manual +PDA - Qatar: manual (planning to use PDA); UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  4. - Lebanon: manual questionnaire with data entry on oracle) -1 country : Manual data entry and OMR ( - Kuwait-OMR was used in census 1995) - 1 country: ICR (- Egypt) - 1 country: Manual data entry and internet (- Palestine: manual; CSPro 3.3 for census data entry/editing; - Oracle for buildings & housing units) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  5. - 1 country- Manual data entry, OMR, OCR/ICR & PDA ( - Morocco) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  6. Question 2: If your office uses optical data capture: i. what are the features of the scanner in use? (e.g. pages per-minute, resolution, type), • 3 countries replied (- Egypt: Fujustu scanner high speed-dropout color (16 doullex/mim, - Kuwait: OPScan 7 & OPScan 10; speed 500 sheets/hr with validation; 5000sheets/hr with no validation ) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  7. Morocco: 40 to 85 page per minutes depending on resolution, feeding orientation and documents size. Resolution 200 or 300dpi. Documents input: max 66 x 30.5. Feeding capacity: 250). UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  8. ii.  What kind of scanning software does your office use (e.g. Softrac, Kodak, Scamax,)? • 3 countries replied ( - Egypt:Use TwAin dirver in application prog, - Kuwait:Scantools , - Morocco: Kodak ds Digital Science Scanner 3520.) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  9. Question 3: Does your office outsource part or all of the data capture/editing process? • 7 countries – replied No • 2 country – replied Yes (Egypt, Morocco) i. If yes, what part of the process is outsourced? - Egypt: Scanning & Auditing Data processing; - Morocco: 50% of data operators are temporarily hired. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  10. ii. If not, what are the methods your office uses for data capture? • 6 countries replied (- Jordan & Kuwait, Qatar & Syria: traditional method, - Palestine- Manual data entry; PDA as pilot, - Iraq- Using db system like Dbase, Foxpro, Oracle: Data entry done in COSIT in addition to decentralized data entry). UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  11. Question 4:What data capture methods does your office plan to use for the next census? •  7 countries replied: (- Jordan: hasn't decided yet, - Kuwait, Palestine & Qatar : PDA in 2010 census, - Iraq: Scanning census questionnaires, - Morocco: OCR/OMR. - Syria: Paper questionnaire) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  12. Question 5: Does your office plan to use optical data capture for the next census/survey? •  8 countries replied: • 5 countries: No (Kuwait, Lebanon, Palestine, Qatar & Syria); • 2 countries: Yes (Iraq & Morocco); • 1 country: May be (Jordan) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  13. i. If so, what technology? •  2 countries replied - (Iraq: ICR/OMR - Morocco: OCR/OMR) ii. If so, will you outsource part or all the process? Which part(s) will be outsourced? • 1 country replied: (Iraq- all the process) iii. If in-house, how many staff work on data capture? • 1 Country replied: ( Morocco- 90 fulltime & 70 ad-hoc) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  14. Question 6: What kind of concerns does your office have with data capture? • 8 countries replied (- Egypt: Archiving according to application needs, - Jordan: Planning and decision making; to help achieving sustainable development, - Kuwait: speed; accuracy and valid data on time for dissemination, UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  15. - Iraq: Accuracy & Timing, - Morocco: Processing census questionnaires required large investment, which is not fully used once the operation is completed, - Palestine: Internet; PDA, Data consistency, Data quality, - Qatar: Quality and Timing, - Syria: Quality.) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  16. Question 7: What are the archiving methods/policies used for the storage of paper forms and electronic forms? • 7 countries replied: (- Jordan : well equipped areas for all hardcopy forms which usually damaged 10 years after census; - Lebanon: Archiving the questionnaires and the data in the oracle db; UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  17. - Kuwait: well organized storage, wooden boxed, serially numbered. E forms kept in db & backups; - Palestine : E. format (oracle db), Papers as per decree issued by Council of Ministries; - Iraq: E. Methods; - Qatar: Conventional papers storage; - Morocco: Archiving paper questionnaires for a period not less than 60 years; questionnaires processed using OCR/ODMR are stored in hard drives and DVDs.) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  18. Summary points on Data Capture • Manual data entry method is used by most of the countries; • Optical data capture method is used by 2 countries; • Software, and good scanners are available in the countries that use optical data capture method . UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  19. Summary points on Data Capture (cont’d) • Outsourcing of data capture and data editing is not applied in most of the countries. • Optical data capture for the next census/survey has not yet been decided by most of the countries, however few countries mentioned that they are planning to use ICR/OCR/OMR. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  20. Summary points on Data Capture (cont’d) • Some concerns were pointed out by some countries regarding data capture such as accuracy, quality and timely dissemination of data. • Archiving methods/policies are used including storage of paper forms and e. forms. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  21. Part C:DATA EDITING • Question 8: Does your office use coding for major classifications of occupations, industry and education? • 9 countries replied Yes, as follow: UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  22. * Planning to use.

  23. 8i. If coding is done, is the method manual or automated or a combination?. • 9 countries replied. (- Jordan, Kuwait, Palestine, Qatar, Iraq and Syria: manual method, - Egypt:automated, - Lebanon & Morocco: combination.) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  24. 8ii. If automated, then what software application is used (whether developed in-house or commercial)? • 3 countries replied (- Egypt:in house, -Lebanon: Oracle, - Morocco: Developed by private contractor.) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  25. Question 9: Does your office have an editing system as part of the census/survey processing? • 9 countries replied Yes. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  26. 9(a).If yes, what kind (if any) of error detection system is used?(e.g., validity checks for all items individually, within record consistency checks, across record consistency checks, macro tabulation checks) (- Egypt:validation in db & all records , - Jordan & Qatar: All of the above, - Kuwait: Batch prog., check data for data class, type, range.. ,then relational validation checks) UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  27. -Lebanon: Validity checks for all items individually, with record consistency checks, across consistency checks , macro tabulation checks. - Morocco:Validity check, record consistency check are used while data is being processed by the OCR, normal video coding and formula video coding. -Palestine-Application is designed to detect errors over item level, record level; validity check, consistency & independency check, UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  28. - Syria:  Record consistency check, - Iraq: 2 stages: i. During data entry sessions using validation rules; ii. after data entry using additional validation rules by different processes like merging files to find/resolve outliers. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  29. 9(b).If yes, what kind (if any) of imputation method is used? (manual, cold deck, hot deck, method based on Fellegi-Holt principle, other methods) • 7 countries replied - Egypt:other methods, - Jordan: not used, - Palestine, Qatar & Syria : manual, - Morocco: Cold deck and hot deck were used in Census 2004, - Iraq-Creating specific automated routines using stat tools (SPSS,STATA) & batch editing tools. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  30. 9(c).If there is an editing and imputation system, what software is used for this purpose? (Developed in-house, CSPRO, other commercial or free software, ex.) • 5 countries replied - Lebanon :Oracle/SPSS, - Morocco: IMPS, CSPRO, - Palestine- CSPro 3.3, - Syria – CSPro, - Iraq-CSPro, Foxpro, SPSS, STATA . UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  31. Summary points on Data Editing • Coding for major classifications such as ISCO for occupation, ISIC rev3 for industry and ISCED for education are used by most countries; • Manual coding method is a common method in most of the countries ; • Countries apply various editing systems such as validity checks, record consistency and detection of outliers. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  32. Part D: Staff Training • Question 10: How many census staff are involved in each of the steps of data processing in your office? Please include both regular NSO staff as well as contractual staff. • 7 countries replied, as follow: UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  33. Number of census staff involved in data capture

  34. Number of census staff involved in data Coding

  35. Number of census staff involved in Error Detection

  36. Number of census staff involved in Imputation

  37. Question 11:For how long does the training last for each of the component steps? • 8 countries replied, as follow: UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  38. Question 12:Does your office implement a set of quality control procedures in relation to the various steps of data processing? • 9 countries replied ( 8 countries: Yes, one country: No -Kuwait) - Egypt : Yes UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  39. Jordan- Yes - Field and office editing for the range, consistency and coverage; - On-line verification for the range, consistency and coverage; - Designing special data entry programs for error detection ; - Designing tables of frequencies for reviewing the results, their consistency and accuracy ; UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  40. - Designing duel tables for reviewing accuracy and consistency of the results and the inter- relations of questions with each other ; - Designing the tables of final results , checking the data of each table and the other relevant tables for consistency; - Reviewing the results for consistency and accuracy prior to dissemination; - Conducting a seminar for discussing the results. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  41. Lebanon- Yes :Special team for verification of performance : tasks- establish a quality standards of a desired level for each of the operation, including manual editing, coding, data entry and computer editing.) Morocco- Yes : Producing data with a minimum accepted error rate, systems and methodologies in accordance with international norm are applied, questionnaires processed using other methods than OCR/OMR such as CSPro are subject to double keying. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  42. Palestine- Yes :(Once a district data is captured, and cleaned, data is distributed to subject matter departments (i.e. Labor, Education, Demography, … etc) to validate the data in terms of consistency, comparability. New checks are added to editing application according to the feed back of subject matter departments. Qatar- Yes: Double keying, verification of data keyed from paper. Syria- Yes: using SPSS; Iraq- Yes : Field training. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  43. Summary points on Staff Training • Staff involved in data capture are more compared with other data processing steps such as data coding, error detection and imputation. • Countries implement quality control procedures for various data processing steps. UNSD-ESCWA Regional Workshop on Census Data Processing in the ESCWA region Doha, 18-22 May 2008

  44. Thank you.

More Related