Developing reliable automatic metadata generation feedback from matdl pathway
This presentation is the property of its rightful owner.
Sponsored Links
1 / 13

Developing Reliable Automatic Metadata Generation: Feedback from MatDL Pathway PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

Developing Reliable Automatic Metadata Generation: Feedback from MatDL Pathway. NSDL Annual Meeting , Washington, DC November 6-8 2007 Advancing NSDL Networks. Cathy S. Lowe, Laura M. Bartolo, Kent State University. Outline. MatDL Pathway

Download Presentation

Developing Reliable Automatic Metadata Generation: Feedback from MatDL Pathway

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Developing reliable automatic metadata generation feedback from matdl pathway

Developing Reliable Automatic Metadata Generation: Feedback from MatDL Pathway

NSDL Annual Meeting , Washington, DC

November 6-8 2007

Advancing NSDL Networks

Cathy S. Lowe, Laura M. Bartolo,

Kent State University


Outline

Outline

  • MatDL Pathway

  • iVia metadata generation for PDFs & test set

  • Evolution of result set

    • description

    • title

    • keywords

    • author

  • Next Steps

NSDL Annual Meeting 2007 Washington, DC


Developing reliable automatic metadata generation feedback from matdl pathway

NSDL Materials Digital Library Pathway

http://matdl.org/matdlwiki

http://matdl.org/virtuallabs

NSF MS Initiatives

(NIRTs, MRSECs, IMIs)

  • Soft Matter Wiki

Virtual Labs

  • Intro to Solid State Chem

  • Intro to Bio Physics

  • Modern Chemistry

Code Development

  • Matforge

    • NIST FiPy

    • CMU

    • DOE CMSN

Teaching Resource

Development

  • MS Teaching Archive

Stewardship

  • MatDL Repository

http://matdlforge.org

http://teaching.matdl.org

http://matdl.org

NSDL Annual Meeting 2007 Washington, DC


Ivia metadata generation original test set

iVia metadata generation & original test set

  • Worked with iVia metadata generation only

  • Test set

    • PDF format

    • 83 undergraduate research papers from Cornell Center for Materials Research (CCMR) REU program

NSDL Annual Meeting 2007 Washington, DC


Developing reliable automatic metadata generation feedback from matdl pathway

NSDL Annual Meeting 2007 Washington, DC


Developing reliable automatic metadata generation feedback from matdl pathway

NSDL Annual Meeting 2007 Washington, DC


Evolution of result set

Evolution of result set

  • Metadata generation for PDFs not available (2005)

  • Metadata generation for PDFs available (2006) – improving over time

    • description

    • title

    • keyword

    • author ** recently available

NSDL Annual Meeting 2007 Washington, DC


Description generation

Description generation

Good accuracy for explicit “Abstract”

  • Correct - ~38%

  • Partially correct – ~33%

  • Incorrect/not generated – ~29%

NSDL Annual Meeting 2007 Washington, DC


Title generation

Title generation

Very good accuracy

  • precision 91.09%

  • recall 89.30%

NSDL Annual Meeting 2007 Washington, DC


Keyword generation

Keyword generation

Manually rated 5 keyphrases per document – Good accuracy

  • Highly descriptive - 39%

  • Acceptable - 41%

  • Unacceptable - 20%

NSDL Annual Meeting 2007 Washington, DC


Author generation new functionality

Author generation --new functionality

Applied to original sample:

  • Correct - 45%

  • Partially correct - 27%

  • Incorrect/not generated - 28%

NSDL Annual Meeting 2007 Washington, DC


Next steps

Next Steps

  • Collaboration mutually beneficial for tool developers & NSDL community-based repositories

  • Continue to work with tool as it improves

  • Continue/expand working with MRSECs REU resources

NSDL Annual Meeting 2007 Washington, DC


Thank you questions clowe@kent edu

Thank you & [email protected]

The NSDL Materials Digital Library Pathway is supported by the National Science Foundation DUE-0532831. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF.

NSDL Annual Meeting 2007 Washington, DC


  • Login