citations as indicators in ts n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CITATIONS AS INDICATORS IN TS PowerPoint Presentation
Download Presentation
CITATIONS AS INDICATORS IN TS

Loading in 2 Seconds...

play fullscreen
1 / 22

CITATIONS AS INDICATORS IN TS - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

CITATIONS AS INDICATORS IN TS. Daniel Gile daniel.gile@yahoo.com www.cirinandgile.com. AIMS OF THIS PRESENTATION. THIS PRESENTATION IS OFFERED IN THE FRAMEWORK OF RESEARCHER TRAINING RESEARCH CONTEXT IS: RESEARCH ON TS AS AN OBJECT OF STUDY

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'CITATIONS AS INDICATORS IN TS' - wauna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
citations as indicators in ts

CITATIONS AS INDICATORS IN TS

Daniel Gile

daniel.gile@yahoo.com

www.cirinandgile.com

D.Gile citations as indicators

aims of this presentation
AIMS OF THIS PRESENTATION

THIS PRESENTATION IS OFFERED IN THE FRAMEWORK OF RESEARCHER TRAINING

RESEARCH CONTEXT IS: RESEARCH ON TS AS AN OBJECT OF STUDY

TS IS A COMPLEX ENTITY, HENCE THE USE OF INDICATORS IN ITS STUDY

THE CONCEPT OF INDICATORS IS EXPLAINED

USES OF CITATIONS AS AN INDICATOR IN THE STUDY OF TS ARE EXPLAINED

METHODOLOGICAL ISSUES WHICH ARE RELEVANT TO OTHER EMPIRICAL STUDIES ARE DISCUSSED

D.Gile citations as indicators

indicators
INDICATORS

INDICATORS ARE

MEASURABLE UNITS

(or at least countable units)

WHICH ARE USED TO MEASURE

(or at least detect/identify)

ASPECTS OF ENTITIES WHICH ARE DIFFICULT TO MEASURE DIRECTLY

BECAUSE THEY ARE COMPLEX AND/OR DIFFICULT TO DEFINE OPERATIONALLY

D.Gile citations as indicators

examples of indicators
EXAMPLES OF INDICATORS

- FOR HEALTH: BODY TEMPERATURE, BLOOD PRESSURE…

- FOR NATIONAL WEALTH: GNP

- FOR LEVEL OF CRIMINAL ACTIVITY: NUMBER OF CRIMINAL TRIALS, NUMBER OF COMPLAINTS TO THE POLICE…

- FOR TRANSLATION QUALITY: NUMBER OF ERRORS AND OMISSIONS, NUMBER OF COMPLAINTS FROM USERS

- FOR QUALITY OF TRANSLATOR TRAINING PROGRAM: PROPORTION OF GRADUATES WHO FIND WORK AS TRANSLATORS

D.Gile citations as indicators

important features of indicators 1
IMPORTANT FEATURES OF INDICATORS (1)

- VALIDITY

(DO THEY MEASURE WHAT THEY ARE SUPPOSED TO MEASURE?)

- CONVENIENCE AND COST OF ACCESS

(HOW LONG DOES IT TAKE TO MEASURE THEM? HOW MUCH MONEY/MANPOWER etc. DOES IT COST?)

- EASE/COST OF PROCESSING

(ONCE YOU HAVE THEM, IN ORDER TO DO SOMETHING USEFUL WITH THEM, HOW MUCH EFFORT AND HOW MANY RESOURCES ARE REQUIRED?)

D.Gile citations as indicators

important features of indicators 2
IMPORTANT FEATURES OF INDICATORS (2)

- COVERAGE

(WHAT PROPORTION OF THE RELEVANT PHENOMENON IN THE ENTITY TO BE STUDIED DO THEY COVER?)

- SENSITIVITY

(HOW MUCH OF THE “SIGNAL” OR PHENOMENON THEY ARE SUPPOSED TO PICK UP THEY ACTUALLY DO PICK UP?)

- SELECTIVITY

(DO THEY PICK UP THE TARGET SIGNAL OR PHENOMENON ONLY, OR DO THEY ALSO PICK UP OTHER SIGNALS WHICH INTERFERE WITH THE EXPLORATION PROCESS?)

COMPLAINTS FOR CRIMINAL ACTIVITY? VAL/COV/SENS/SEL

COMPLAINTS OF TRANSLATION USERS? VAL/COV/SENS/SEL

PROPORTION OF GRADUATES WHO FIND EMPLOYMENT? VAL/COV/SENS/SEL

D.Gile citations as indicators

citations
CITATIONS

WHEN INVESTIGATING TS AS AN OBJECT OF STUDY:

WRITING NORMS IN SCIENCE

IMPOSE REFERENCES TO AUTHORS AND WORKS “USED”

THESE TAKE THE FORM OF CITATIONS

Operational definition:

EXPLICIT REFERENCE IN A RESEARCH TEXT TO ANOTHER AUTHOR’S PUBLISHED TEXT

i.e. NAME OF AUTHOR AND PARTICULAR TEXT WITH REFERENCES

D.Gile citations as indicators

citations as an indicator of influence
CITATIONS AS AN INDICATOR OF INFLUENCE

VALID?

EASE AND COST OF ACCESS?

EASE AND COST OF PROCESSING?

COVERAGE?

SENSITIVITY?

SELECTIVITY?

D.Gile citations as indicators

in the field
IN THE FIELD

CONSIDERED VALID

WIDELY USED

By

GOVERNMENTS, SOCIOLOGISTS OF SCIENCE…

IN TS:

COVERAGE INSUFFICIENT

SENSITIVITY PERHAPS INSUFFICIENT

CORRELATION BETWEEN NUMBER OF CITATIONS AND ACTUAL INFLUENCE PERHAPS NOT STRONG ENOUGH

D.Gile citations as indicators

tentative methodological improvement 3 levels of influence of a cited text
TENTATIVE METHODOLOGICAL IMPROVEMENT - 3 LEVELS OF INFLUENCE OF A CITED TEXT?

L1: In passing, as illustration, historical reference,

NO VISIBLE IMPACT on citing author’s thought or work, judging by the citing text

L2: SOME IMPACT: Clear signs in citing text that citing author has thought about the cited text significantly or is building on it

L3: CENTRAL in citing text

EVALUATE INFLUENCE OF CITED TEXTS IN CITING TEXT

D.Gile citations as indicators

3 levels of influence is it a progress
3 LEVELS OF INFLUENCE: IS IT A PROGRESS?

PERHAPS TOO MUCH VARIABILITY?

PERHAPS DIFFICULT TO CATEGORIZE?

HAVE TO CHECK

IF NOT GOOD ENOUGH, IMPROVE DEFINITION

IF NOT GOOD ENOUGH, GIVE IT UP

CHECK IN CLASS WITH PAPERS

SEE % OR AGREEMENT, DISAGREEMENT

IF OK, COMPARE RESULTS WITH 3 LEVELS AND WITHOUT

D.Gile citations as indicators

3 levels of influence tests
3 LEVELS OF INFLUENCE: TESTS?

A NUMBER OF TESTS WITH COLLEAGUES

TAKE PUBLISHED TEXT

ASK PANEL OF RESEARCHERS TO ASSESS LEVEL

CHECK CONVERGENCE

IF CONVERGENCE, OK, IMPLEMENT

(IF COLLECTION COST NOT TOO HIGH)

IF NOT ENOUGH CONVERGENCE

- TRY TO IMPROVE OPERATIONAL DEFINITION

- OR INFER THAT THIS IMPROVEMENT, HOWEVER ATTRACTIVE CONCEPTUALLY, DOES NOT REALLY WORK

D.Gile citations as indicators

operationalizing citation analysis to measure author influence
OPERATIONALIZING CITATION ANALYSISTO MEASURE AUTHOR INFLUENCE

- DETERMINE SAMPLING METHOD AND SAMPLE SIZE

(TYPE OF CITING TEXTS, PERIOD, LANGUAGES, GEOGRAPHIC AND SOCIOLOGICAL IDENTITY…)

IF THESE CONSIDERATIONS ARE NOT TAKEN UP, PROBLEM OF REPRESENTATIVITY

  • PROCESSING

(SIMPLE COUNTS? PROPORTIONS? NUMBER OF TIMES CITED OR NUMBER OF PAPERS IN WHICH CITED?...)

D.Gile citations as indicators

what questions can citanal help with initial contributions
WHAT QUESTIONS CAN CITANAL HELP WITH?INITIAL CONTRIBUTIONS

- WHO ARE THE MOST INFLUENTIAL AUTHORS AT TIME t?

IN TS? IN A PARTICULAR BRANCH OF TS? ON A PARTICULAR TOPIC? WITH A PARTICULAR METHODOLOGY?

- SAME QUESTION ABOUT GROUPS, “SCHOOLS”

- EVOLUTION OF INFLUENCE OVER TIME

- INFLUENCE OF PUBLICATION MEDIUM

(PAPERS IN JOURNALS, PAPERS IN COLLECTIVE VOLUMES, MONOGRAPHS, THESES AND DISSERTATIONS)

- INFLUENCE OF LANGUAGE OF PUBLICATION

D.Gile citations as indicators

what questions can citanal help with assessing quality
WHAT QUESTIONS CAN CITANAL HELP WITH?ASSESSING QUALITY?

ARE CITATIONS A VALID INDICATOR FOR RESEARCH QUALITY?

QUESTION:

ARE THE MOST OFTEN CITED AUTHORS IN TS/IN OTHER DISCIPLINES

THE ONES WHO DO THE BEST RESEARCH?

D.Gile citations as indicators

what questions can citanal help with developing their use
WHAT QUESTIONS CAN CITANAL HELP WITH?DEVELOPING THEIR USE

- INTERACTIONS BETWEEN GROUPS

(INCLUDING INTERDISCIPLINARITY)

COLLECT INFORMATION ON CITED TEXT - FROM TS OR OTHER DISCIPLINES? Gile 2002/2006

TYPES OF CITATIONS

- THEORIES

- FINDINGS

- RESEARCH METHODS

- OPINIONS

- TERMS…

INDICATIONS ON QUALITATIVE ASPECTS OF TS RESEARCH

EMPIRICAL/NON EMPIRICAL

D.Gile citations as indicators

general methodological note thresholds
GENERAL METHODOLOGICAL NOTE:THRESHOLDS

ASSUME “EMPIRICAL”: FINDINGS, METHODOLOGY, REFERENCES TO EMPIRICAL STUDIES

NON EMPIRICAL: TERMS, OPINIONS,

IF 100% “EMPIRICAL” NO PROBLEM

IF 1% “EMPIRICAL” NO PROBLEM

IF 50% “EMPIRICAL”?

IF 30% “EMPIRICAL”?

NECESSARILY SUBJECTIVE/CONVENTIONAL THRESHOLDS

THIS IS COMMON TO MANY EMPIRICAL PROJECTS WITH METRICS SUCH AS PAUSE LENGTHS, NUMBER OF E/O’S, etc.

AT SOME TIME, SUBJECTIVE DECISIONS COME IN

D.Gile citations as indicators

general methodological note classifications
GENERAL METHODOLOGICAL NOTE: CLASSIFICATIONS

FOR MANY USES OF INDICATORS IN EMPIRICAL RESEARCH, CLASSIFICATION IS CENTRAL

PRACTICAL ISSUE:

NOT EVERY PHENOMENON ENOUNTERED IS CLASSIFIABLE:

- MISSING INFORMATION

  • BORDERLINE CASES

WHAT DOES ONE DO?

CONVENTIONS, PERSONAL DECISIONS, INFORM READER

PRACTICAL ASSIGNMENT TO GAIN FAMILIARITY WITH PROBLEMS ?

D.Gile citations as indicators

assignment 1
ASSIGNMENT (1)

CHOOSE AT LEAST 3 SPECIFIC RESEARCH QUESTIONS FOR WHICH YOU BELIEVE CITATIONS ARE A GOOD INDICATOR

Or

THE EXPLORATORY QUESTION: WHAT CAN I FIND OUT ABOUT MY SAMPLE FROM CITATIONS?

CHOOSE APPROPRIATE SAMPLE OF TEXTS TO TRY TO ANSWER THE QUESTIONS

- At least 20 papers with a total of at least 200 citations if specific research questions

- At least 5 papers with a total of at least 50 citations if your question is exploratory

D.Gile citations as indicators

assignment 2
ASSIGNMENT (2)
  • STUDY THE SAMPLE

REPORT ALL THE CITATION DATA ON EXCEL FILE

(SEE SUGGESTIONS LATER IN THIS PRESENTATION)

  • REPORT RESULTS AND ANSWER TO QUESTION(S) ON SEPARATE WORD FILE

- REPORT CAREFULLY ALL METHODOLOGICAL PROBLEMS YOU ENCOUNTERED AND ALL METHODOLOGICAL LESSONS YOU LEARNED FROM THE EXERCISE

D.Gile citations as indicators

suggestions for columns of info on data excel file 1
SUGGESTIONS FOR COLUMNS OF INFO ON DATA EXCEL FILE (1)

1. Citing author

2. (Affiliation)

3. Year of citing paper

4. References of citing paper

5. Page in which citation found

6. Cited author

7. (Affiliation of cited author)

8. (Discipline of cited author)

9. Year of cited reference

10.Cited reference

D.Gile citations as indicators

suggestions for columns of info on data excel file 2
SUGGESTIONS FOR COLUMNS OF INFO ON DATA EXCEL FILE (2)

11. Type of medium (thesis, dissertation, article, article in collective volume, monograph, internet document…)

12. Language

13. Year of cited reference

14. Cited reference

15. Type of reference (opinion, definition of concept, term, finding, methodology, reference to empirical study, plain reference to publication, theory,…)

16. Other comments or key words

D.Gile citations as indicators