After all the coding is done ...

Presentation Transcript

Harry Ganzeboom

Center for Survey Research – Academia Sinica

July 24-25 2008

Scaling occupations

Detailed occupation codes have various uses, but for most applications they are condensed again into social status scales.
There is a great variety of national and international social status scales and ways they are constructed.
Main division:

Nominal categories: EGP (Goldthorpe), Wright, Esping-Andersen.
Continuous scales: Prestige, Socio-economic Index [SEI]

Each of these have their own theoretical backgrounds.
The varieties of social status scales can only be compared when you have access to detailed occupations (and more).

Analysing occupational information

Tools for ISCO-88

http://home.fsw.vu.nl/HBG.Ganzeboom/ISMF
This webpage contains several useful [SPSS] tools to work with ISCO-88 codes:

ADD VALUE LABELS for all occupations
RECODE for EGP social classses
RECODE for SIOPS [Treiman’s] prestige scale
RECODE for ISEI [Ganzeboom et al.’s] SEI scale

Note that the tools will work (A) for multiple occupations, and (B) for all levels of detail of coding (providing you have used trailing zeroes).
There are also tools for ISCO-68 and will be for ISCO-08.

ISEI (1)

A SEI [socio-economic index or Duncan] score scales occupation by averaging status characteristics of job holders, most often their education and earnings.
Often the criterion information is taken from census data.
ISEI was created for ISCO-88 using criterium information for educational and earnings ranks on a ‘world-wide’ sample of 70.000 men from 17 countries.

ISEI (2)

ISEI was constructed as an optimal scaling of (detailed) occupations as an intervening variable between education and earnings: “Occupation is what you do to convert your qualifications into income”.
Metric between 10-90, but this is entirely arbitrary.
ISEI was originally developed for ISCO-68, but its second generation version (for ISCO-88) has become widely used, also outside sociology.

Prestige

Prestige: popular evalation of occupational status, i.e. you ask respondents to value occupations.
Many local versions have been integrated by Treiman (1977) into the Standard International Occupational Prestige Score [SIOPS], related to ISCO-68.
The version on my website is a mapping of the original SIOPS to ISCO-88.

EGP

EGP class typology combines detailed occupation codes with measures on self-employment and supervising status.
This leads to a nominal (partly ordered) set of distinctions: 12-10-7-5 categories.
EGP has become the de facto standard for stratification research. Much used.

Relationships EGP, ISEI, SIOPS

All these measures are strongly associated. You need a lot of data if you are going to argue about the differences.
EGP and ISEI resemble each other more than SIOPS.
SIOPS [prestige] is theoretically the best idea, but it does not work well in practice.
I prefer to use ISEI for my further discussion here.

Checks to be run ...

Use value labels to see whether the coders have indeed entered only valid codes.
It is surprising to learn how often this check has not been run!
It is even more surprising to learn how often this is the only check ever run!!!

MTMM-models

Multi-Trait Multi-Method models were developed in psychometrics to estimate the reliability and validity of attitude items.
The idea is that you can learn about reliability and validity (both!!) when you apply multiple methods (e.g. respons formats) to multiple [related] traits (e.g. personality characteristics.
Remember:

Reliability: lack of random errors
Validity: lack of systematic error

MTMM model

ROCC

FOCC

FISEI1

FISEI2

RISEI1

RISEI2

Estimating MTMM for two coders

The elementary MTMM model for two traits (occupations) and two methods (coders) has 7 parameters.
The data generate only 6 degrees of freedom.
However, by contraining (equalizing) the parameters, we can find the following interesting information:

How random error each coder has coded relative to the other.
Whether FOCC and ROCC differ in the amount of random error.
How much systematic bias each coder has added to their codes.
Degree of attention brought about by the coding unreliability; corrected (disattenuatud) correlation between FOCC and ROCC.

What are we learning by staring at these correlations?

Within-coder correlation at best 0.81. This means 0.90 index of reliability.
Coders agree slightly less on father’s occ than respondent’s. Loss is around 0.97.
Within- and cross-coder intergenerational correlations are around 0.33 and fairly homogenous.
Coder 1 has created slightly more consistency between father and respondent.

MTMM assumptions

Coders are equally reliable for fathers and respondents.
However, fathers’ occupations may be easier to code (less) reliably than respondents’ occupations.
Systematic error is the same for all coders.

If estimated by SEM (Lisrel), we learn:

Reliability coder 1/coder 2: 0.915 / 0.886 (NS).
Reliability FOCC/ROCC: 0.975 / 1.000 (NS).
Coder unique consistency: 0.015 (significant).
Corrected intergenerational correlation: 0.413.

The interesting conclusion for this (Italian) example is clearly the corrected intergenerational correlation. Note that this is even so with high coder reliability!

Conclusions

Even if coders do a decent and honest job, they introduce random and systematic error.
These errors are in the coding process, not by the data collection!
If coders introduce only 10% error, they bring down the intergenerational correlation by 20%!

More sources of measurement problems .. and their repairs

It is important to see that coder errors are just one single source of bad measurement.
It might be true that even bigger trouble is created by what the respondents say.
If you want to assess measurement error at the respondents level, you need to ask the question twice:

Within the same interview
From different sources (e.g. spouses about each other).
At diffent interviews, e.g. in panel designs.

Another source of error: the respondent.

Note that all of the above is about errors generated in the coding proces.
Occupational measures also contain other errors, most prominently generated by the respondent / interviewer.
This type of error can only be estimated by asking the question again:

In the same interview.
From a different source (e.g spouses about each other).
In a different interview (panel).

Can you ask the occupation question again in the same interview?

Yes, an acceptable way for respondents is to ask an open question (see above) and a closed question.
Closed questions may not be as valid and flexible as open questions, but they may be more reliable. At least they do not suffer from coding error...
This type of multiple measurement has been tried in ISSP87 for four countries and six Dutch surveys. It will be replicated in ISSP09.

Main conclusions on double measurement

Crude closed questions are slightly more reliable than detailed open question.
Crude questions suffer slightly more from systematic error than detailed questions:

Correlated error (‘echo effects’)
Education bias.

However, the main boost comes from using multiple indicators, that leads to disattenuation. Estimates from ISSP and Dutch data suggest measurement relationships of around 0.85. This would suggest that coding error is the major source of random error.

ISCO 2008

ILO has recently revised the ISCO to ISCO-08.
Current situation is that the new classification has been fixed and published.
However, there are no definitions or manuals available yet.
For previous versions it laster 1-2 years before these became available.

Stated goals of ISCO-08

Bring occupational classification in line with changed technologies and division of labor (e.g. ICT/IT).
Make ISCO applicable in a wider range of countries and economies.
To mend often noted problems with the application of ISCO-88.
To produce a minor revision, not a totally different classification.

Problems with ISCO-88 (1)

Unlike its predecessor (ISCO-68), ISCO-88 is primarily skill oriented. However, in practice the major group differentiation does not closely correspond to major ISCED (education) levels.
ISCO-68 was more sensitive to employment status (self-employment) and industry.

Despite its stated principles, it is hard to pay tribute to skill level differentiation in manual work. ISCO-88 differentiates between (7000) Craft workers, and (8000) Machine Operators, which is similar, but not the same as Skilled versus Semi-skilled Manual Workers.
In addition, many occupations occur both in the 7000 and 8000 categories.

ISCO-88 argued that occupation and employment status are different things and need to be measured separately.
As a consequences some employers became classified with their employees, in particular there is no distinction between managing proprietors and managers, and not between working proprietors and their employees.

Managers were organized into three levels:

Corporate managers
Department managers [Production, Support]
General [Small enterprise] managers.

The primary distinction here is the number of managers in an organisation, which is not often available in data.
It is somewhat hard to classify work supervisors [Foremen] in ISCO-88.

Farmers are hard to classify in ISCO-88, because they appear in 5 places:

Operations Department Manager (1211)
Small Establishment Manager (1311)
Skilled Agricultural Worker (6100)
Subsistence Farmer (6200)
Farm Laborer (9200)

None of this corresponds closely to distinctions made in farm work in national classifications.

ISCO-88 is overly broad in (5000) Service and Sales Occupations.
In particular (5200) Sales Workers is very undifferentiated.

It is hard to find fitting codes for ‘crude’ occupations: factory worker, skilled worker, foreman, semi-skilled worker, apprentice.
However, in some instances, there is no problem if one used major and sub-major groups codes: e.g. (9000) for Unskilled Worker.

ISCO-08 groups

10 major

34 sub-major

120 minor

403 unit

Total: 567 groups

ISCO-88 groups

10 major

28 sub-major

115 minor

363 unit

Total 516 groups

ISCO-08 versus ISCO-88

Mergers and Splits

Mergers: Many-to-one recodes.
Splits: One-to-one recodes.
Mergers & splits: Many-to-many recodes.
All of these occur when comparing ISCO88 to ISCO08.
When we crosswalk from 88 to 08 (and have no further information), only mergers are relevant.
When we have ISCO88 and further information (like original verbatim info of original source classification), we also need to consider splits.

Mergers

Splits

Major groups

10 major groups: Essentially unchanged, with minor changes of titles.
However: If minor groups have been moved between major groups (see below), this de facto changes major groups too!
The major group that is likely most affected by such shifts is (5000) and in particular (5200) Sales Workers, that now contains a number of Elementary Sales Occupations.

Sub-major groups (2 digits)

34 sub-major groups: expanded from 28 major groups.
Truly NEW:

(0100, 0200, 0300) Army ranks (3x)
(9400) Food Preparation Workers

Other ‘new’ major groups are ‘upgraded’ or ‘merged’ minor groups. Roughly speaking, about half of the sub-major groups has remained the same, the other half has a different composition than in 1988.

ICT occupations

Altogether, ISCO-08 distinguishes ca. 20 ICT occupations, that occur at several levels:

(2500) ICT Professionals (11x)
(3500) ICT Technicians (5x)
(1330) ICT Service Manager (1x)
(2356) ICT Teachers (1x)
(2434) ICT Sales Professionals (1x)

Neither (2500) nor (3500) are new – actually both existed already in ISCO-68!

Problem 1: Imperfect skill orientation

Some ambiguities between (7000) Craft Workers, and (8000) Machine Operators have been removed.
An NEW feature is the distinction between (8100) Stationary Machine Operators, and (3130) Process Control Technicians, which probably refers to the complexity of the process / machine controlled / operated.

Problem 2: Employment status

Although somewhat indirect, ISCO-08 has better fitting codes for Large Entrepreneurs and Foreman.
There is an ambiguous distinction between (1420) Retail and Wholesale Trade Managers, and (5221) Shop Keepers.

Problem 3: Managers

The implicit reference to firm size (i.e. number of departments) has disappeared, the same things are now referred to by main activity.
At the sub-major group level Corporate Managers are now longer grouped with department managers, but with (high) Government Officials.
Major changes occur at the 3-digit and 4 digit level.

(1330) ICT Services Managers
(1340) Professional Services Managers (9x)

Problem 4: Farmers

Self-employed farmers can still be coded in as (1310) Managers in Agriculture etc.
However, it also remains possible to code them with (6100) Market-oriented Skilled Agricultural Workers.
Interestingly, a NEW feature is that (6200) Subsistence Farmers has now four minor groups.

Problem 5: Crude Sales / Service

Sales salespersons are split:

(5221) Shop Keepers
(5222) Shop Supervisors
(5223) Shop Sales Assistants

This is an improvement.

Also, more levels and locations of sales (market, stall, cashiers) have been regrouped in the sub-major group (5200).

This has made the sub-major group even more heterogeneous than it was.

Interesting ..

Cooks are now split up into

(3434) Chef [a “Culinary Associate Professional”]
(5120) Cooks
(9400) Food Preparation Workers

(9411) Fast Food Preparers
(9412) Kitchen Helper

I am very happy with this...

Problem 6: Crude occupations

Some of the new features mend this problem:

“Foreman” can now be classified as (3120) Production Supervisor.
“Shop keeper” can go in two places.
“Skilled Worked” can be more conveniently coded as (7000).

Interesting ...

Specialized Secretaries and Office Managers are now in (3000) Associate Professionals.
Some new occupations:

(2230) Traditional and Complementary Health Professional
(5245) Service Station Attendant
(7234) Bicycle Repairman
(9334) Shelf Filler
(9412) Kitchen Helper

Disappeared:

(2121) Mathematician, Statistician
(6142) Charcoal Burner

How can we reclassify existing data?

A simple conversions of ISCO-88 into ISCO-08 is not possible.
Conversion tool will become available, that will do two things at the same time:

Straight recode of ISCO-88 into ISCO-08 (‘best fit’). Truncate trailing decimals, if this is the only thing that you want or can do.
Trailing decimals suggest the amount of alternatives (splits). You will have to consult a separate document to list these options. For this to be usefull you will need original strings or classifications.

After all the coding is done ... - PowerPoint PPT Presentation

PowerPoint Slideshow about 'After all the coding is done ...' - HarrisCezar

After all the coding is done ...