Data delivery
Download
1 / 19

summary - PowerPoint PPT Presentation


  • 192 Views
  • Updated On :

Data delivery Eileen Howes 10 April 2006. Summary. What we wanted What we got What we want from 2011 Census. What we wanted. Test data Early release of data for pre-processing To help with QA as data experts CSV files Data that was correct first time Table layouts as agreed in advance.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'summary' - richard_edik


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Data deliveryEileen Howes10 April 2006


Summary l.jpg
Summary

  • What we wanted

  • What we got

  • What we want from 2011 Census


What we wanted l.jpg
What we wanted

  • Test data

  • Early release of data for pre-processing

  • To help with QA as data experts

  • CSV files

  • Data that was correct first time

  • Table layouts as agreed in advance


Test data l.jpg
Test data

  • Got some eventually

  • Had to fight for it


Early release for pre processing l.jpg
Early release for pre-processing

  • No

  • Got the Supertable data on publication date

  • Had to wait a bit longer for CSV files

  • Thousands of users had to wait a lot longer


User help with qa l.jpg
User help with QA

  • No

  • So found the errors as soon as we loaded the data


Csv files l.jpg
CSV files

  • Some Supertable

  • Some CSV files

  • Some Excel spreadsheets


Data that was right first time l.jpg
Data that was right first time

  • Not always – but a lot to ask

  • Expert users would have found some of the errors but not all


Table layouts as agreed in advance l.jpg
Table layouts as agreed in advance

  • No

  • Some were as published

  • Others not known until the data arrived

  • So even more delays in processing

  • Some were different for different areas


A few points l.jpg
A few points…

  • Almost all the right numbers

  • Not necessarily in the right order

  • Some re-releases of data – extra work but generally OK

  • Re-releases of just the cells that were wrong –please, NO


A few more points l.jpg
A few more points…

  • Commissioned tables

  • Some files with no area codes

  • Because we didn’t specify them

  • But we expected them to be there


Nightmare at city hall l.jpg
Nightmare at City Hall

  • Commissioned table C0310 in Supertable

  • Output area of workplace to ward of residence

  • Part of SWS Table 301, means of travel to work

  • ONS offered to split large file for us


Nightmare at city hall13 l.jpg
Nightmare at City Hall

  • To export csv files from Supertable:

  • Tried to export csv but computer crashed

  • So ran it until it crashed, retrieved it it parts

  • Started next run from where it crashed


Nightmare at city hall14 l.jpg
Nightmare at City Hall

  • CSV file with 15,200,000 records

  • Then had to add area codes

  • Area names already there


And another thing l.jpg
And another thing…

  • Area names were not always there

  • But if the names were not there

  • Then the codes were

  • Extra work adding them back in

  • We need both


What we want from 2011 census l.jpg
What we want from 2011 Census

  • Standard datasets

  • Nationally comparable data

  • Test data

  • Early release of data for pre-processing

  • To help with QA as data experts

  • CSV files

  • Data that is correct first time

  • Table layouts as agreed in advance


What we want from 2011 census17 l.jpg
What we want from 2011 Census

  • ONS to ditch Supertable

  • Use something that produces usable csv files

  • Is easy to use

  • Is easy to print from


What we want from 2011 census18 l.jpg
What we want from 2011 Census

  • Basic standards ALWAYS adhered to

  • Unique area codes always included

  • Area names always included

  • All numbers in the same order for different areas

  • Agreed table layouts not changed at the last minute


What we want from 2011 census19 l.jpg
What we want from 2011 Census

  • Better method of disclosure control

  • Rounding only if pre-tabulation

  • Consistent database

  • Data for administrative areas – wards and parishes

  • etc


ad