Human resources outcome indicators approaches and prospects
Download
1 / 17

Human Resources Outcome Indicators: Approaches and Prospects - PowerPoint PPT Presentation


  • 517 Views
  • Updated On :

Human Resources Outcome Indicators: Approaches and Prospects Colloquium on Measuring the Impacts of Science June 17-18, 2004 Montreal, Canada Rolf Lehming <rlehming@nsf.gov> Division of Science Resources Statistics, National Science Foundation

Related searches for Human Resources Outcome Indicators: Approaches and Prospects

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Human Resources Outcome Indicators: Approaches and Prospects' - Faraday


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Human resources outcome indicators approaches and prospects l.jpg

Human Resources Outcome Indicators:Approaches and Prospects

Colloquium on Measuring

the Impacts of Science

June 17-18, 2004

Montreal, Canada

Rolf Lehming <rlehming@nsf.gov>

Division of Science Resources Statistics, National Science Foundation

NOTE: The views expressed are those of the author and do not represent the position of the National Science Foundation. No endorsement by the National Science Foundation should be inferred.


Slide2 l.jpg

Two types of research output

  • New knowledge in the form of research results, methodological advances, conceptual breakthroughs

  • People who have gained substantive, conceptual, methodological and tacit knowledge along with a perspective on science and research

  • Both can theoretically be traced and analyzed

    • Bibliometrics as knowledge output proxy

    • Graduate enrollment, advanced degrees, and postdocs as proxies

Human Resources Outcome Indicators


Slide3 l.jpg

Longer-term research outcomes

  • Individuals’ research experiences and encounters with science may yield outcomes like

    • An appreciation of the scientific perspective

    • An openness to empirical inquiry

    • The acquisition of specific skills

  • The sum of these individual outcomes may yield

    • A technically sophisticated population and workforce

    • An educated citizenry able to deal with the science and technology components of public policy issues

Human Resources Outcome Indicators


Slide4 l.jpg

NSF Strategic Goals

  • “People” as one of NSF’s four Strategic Performance Goals

    • (along with “Ideas,” “Tools” and “Administration and Management”)

  • The strategic “People” outcome goal is to develop a workforce of scientists and engineers that is

    • Diverse, internationally competitive, and globally engaged

  • And to develop well-prepared [scientifically literate?] citizens

  • The plan includes annual performance goals and prospective and retrospective reporting foci

Human Resources Outcome Indicators


Slide5 l.jpg

NSF Administrative Data

Number of people involved in NSF’s activities

FY2002 FY2003 FY2004

_____________________Actual Estimated Estimated

Senior researchers 28,960 29,820 30,590

Other professionals 12,060 12,180 12,640

Postdoctoral associates 5,740 6,060 6,170

Graduate students 26,170 27,440 28,690

Undergraduate students 34,250 32,710 36,350

K-12 students 11,460 13,640 14,640

K-12 teachers 84,710 85,460 86,830

Total number of people 203,350 207,310 215,910

Human Resources Outcome Indicators


Slide6 l.jpg

NSF Administrative Data

  • These figures are compiled from proposal budgets estimating number and types of people involved

    • Limited personal information is captured about the principal investigator(s) only

  • NSF asks principal investigators to fill in, as part of project closeout, a matrix of project participants by type of position and demographic characteristics

    • Submission is voluntary and compliance variable

  • Figures do not include the almost 200,000 reviewers of the 30,000-plus proposals NSF receives annually

Human Resources Outcome Indicators


Slide7 l.jpg

NSF Administrative Data

  • The data provide approximate estimates of the number of people “reached,” in some fashion, by NSF funds or, in the case of peer reviewers, by NSF programs

  • These initial project estimates

    • are not exact

    • cannot be aggregated over time (the same persons may be counted in 2, 3, or more adjacent years)

    • in their current form invite no further analytical use

Human Resources Outcome Indicators


Slide8 l.jpg

NSF’s Strategic People Goal

  • NSF discussion about requiring investigators to supply demographic and contact information for project participants to allow use for

    • Tracking of careers

    • Studies of (scientific) productivity

    • Program assessments

Human Resources Outcome Indicators


Slide9 l.jpg

NSF’s Strategic People Goal

  • NSF declined for reasons including.(???)

    • Concern about PI response burden

    • Feasibility and legality (informed consent, university role)

    • Rules for establishing and maintaining Federal Systems of Records

    • Confidentiality and privacy concerns

Human Resources Outcome Indicators


Slide10 l.jpg

NSF’s Strategic People Goal

  • NSF emphasizes HR output and outcomes in proposal guidelines

  • “Proposals must describe … the broader impacts resulting from the proposed activities …”

    • Integration of research and education

    • Participation of underrepresented groups

    • Infrastructure enhancement such as facilities, instrumentation, networks, partnerships

    • Enhancement of scientific and technological understanding

    • Potential benefits of the activity to society at large

Human Resources Outcome Indicators


Slide11 l.jpg

NSF’s Strategic People Goal

  • NSF also emphasizes the importance of HR output and outcomes in reviewer instructions

  • Reviewers are instructed to address both merit review criteria

    • Intellectual merit of the proposed activity

    • Broader impacts [i.e., as described before]

  • Response to the broader-impacts criterion by proposal submitters and reviewers has been uneven

    • NSF has begun returning some proposals

    • NSF has repeatedly urged reviewers to address the second criterion

Human Resources Outcome Indicators


Slide12 l.jpg

NSF Surveys

  • Three biennial surveys of individuals who have at least a bachelor’s degree in science, engineering, or math or work in an S&E occupation

    • National Survey of College Graduates samples people with a bachelor’s or higher degree drawn from decennial U.S. Census, then followed up

    • National Survey of Recent College Graduates samples those with newly awarded S&E degrees

    • Survey of Doctorate Recipients draws a sample from a census of recipients of U.S. S&E doctorates

  • Integrated into the Scientists and Engineers Statistical Data System (SESTAT)

Human Resources Outcome Indicators


Slide13 l.jpg

NSF Surveys

  • Extensive demographic, educational, work-related, and personal information

  • Permits some outcome-related analyses including

    • Expansion of S&E occupations in the labor force (4 times faster growth than total LF over 2 decades)

    • Tracking of S&E degree holders into occupations not classified as science or engineering

      • High degree of relatedness of job, training

      • May indicate growing technical requirements of the economy

  • For individuals, education and career histories

Human Resources Outcome Indicators


Slide14 l.jpg

NSF Surveys

  • Outcome type variables but only inferred link to input, output items

  • Other limitations include

    • Expanded NAICS coding but aggregating up in analyses and barring of firm-level reporting

    • Limited ability to do productivity studies because of the paucity of output indicators

      • Matching with data from external sources faces serious difficulties

    • No ability to address citizenship competency issues

      • Separate survey on public attitudes

Human Resources Outcome Indicators


Slide15 l.jpg

Other NSF Survey

  • Survey of Industrial Research and Development

    • 25,000 firms, 2 personnel items

      • total payroll employment

      • Estimated FTE number of scientists and engineers working on R&D

      • Excludes social sciences and psychology

  • Analyses of the distribution of FTE researchers by firm sector and characteristics including R&D volume

  • In principle, trends in use of researchers across industries

  • No human resources detail available

Human Resources Outcome Indicators


Slide16 l.jpg

Non-NSF Surveys

  • The Bureau of Labor Statistics’ Occupational Employment Survey

    • Tracking of trends in S&E occupations at industry level in the framework of total employment

    • Potential to develop comparable state-level estimates (technical issues to be resolved)

    • Potential outcome indicators on changing size, composition, location of high-technology segments of the economy

      • BLS defines high-tech industries by S&E jobs percentage at least twice that of the average for all industries.

Human Resources Outcome Indicators


Slide17 l.jpg

Bottom Line and a Note

  • These surveys provide some means of estimating the size and some composition of the S&E workforce

  • All surveys but Industrial R&D Survey have only implied link of outcomes to inputs and outputs (except education inputs)

  • Limited ability to conduct in-depth panel analyses of career patterns in the context of the total labor market

  • Limited ability to conduct productivity studies 

  • A note on intergenerational social mobility

    • Valued mark of an open society

    • Education traditionally a strong factor

    • Role of science?

Human Resources Outcome Indicators