1 / 22

Sharing Resources in CLARIN-NL

Sharing Resources in CLARIN-NL. Jan Odijk, Arjan van Hessen LRTS Workshop IJCNLP Chiang Mai, Thailand, 12 Nov 2011. Overview. Context Documentation Visibility Referability Accessibility Long Term Preservation Interoperability Conclusions. Context. CLARIN-NL

gisela
Download Presentation

Sharing Resources in CLARIN-NL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sharing Resources in CLARIN-NL Jan Odijk, Arjan van Hessen LRTS Workshop IJCNLP Chiang Mai, Thailand, 12 Nov 2011

  2. Overview • Context • Documentation • Visibility • Referability • Accessibility • Long Term Preservation • Interoperability • Conclusions

  3. Context • CLARIN-NL • National project in the Netherlands • 2009-2015 • Budget: 9.01 m euro • Funding by NWO (National Roadmap Large Scale Infrastructures) • Coordinated by Utrecht University • 24 partners (universities, royal academy institutes, independent institutes, libraries, etc.)

  4. Context • Dutch National contribution to the Europe-wide CLARIN infrastructure • Prepared by CLARIN preparatory project (2008-2011) • Also coordinated by Utrecht University • From Dec 2011 to be coordinated by the CLARIN-ERIC • ERIC: a legal entity at the European level specifically for research infrastructures

  5. CLARIN infrastructure (NL) • An technical research infrastructure in which a humanities researcher who works with language-related resources • Can find all data relevant for the research • Can find all tools relevant for the research • Can apply the tools to the data without any technical background or ad-hoc adaptations • Can store data resulting from the research • Can store tools resulting from the research via one portal

  6. CLARIN infrastructure (NL) • This requires systematicsharing of resources (=data, tools, web services, …) • Systematic Sharing requires • Documentation • Visibility • Referability • Accessibility • Long Term Preservation • Interoperability of resources

  7. CLARIN-NL subprojects • Resource curation projects • Curate an existing resource • Demonstrator projects • Curate an existing tool and supply a demonstration scenario • #subprojects 21 (12-14 in 2012) • Data Curation Service • Offers the service of curating existing data • Where curationincludes • Documentation, Visibility, Referability, Accessibility, Long Term Preservation, Interoperability

  8. CLARIN-NL Centres • CLARIN infrastructure is virtual and distributed • CLARIN-Centres work together to implement the infrastructure • Each stores and makes available a part of the resources • Some also provide computational facilities • Centres must meet a list of requirements and be certified by CLARIN • Candidate CLARIN Centres in NL • Institute for Dutch Lexicology (INL) • Max Planck Institute for Psycholinguistics (MPI) • Meertens Institute (MI) • Huygens ING Institute (HI) • Data Archiving and Networked Services (DANS)

  9. Infrastructure Implementation • Implementation of basic infrastructure functionality • setting up authentication and authorizations systems • several registries (e.g. ISOCAT, RELCAT, Metadata Registry) • various other infrastructure services • Search Facilities • In resource descriptions (`metadata’) • Centralized after metadata harvesting • In the data themselves • Via federated search • Using Webservices in Workflow systems • Cooperation with Flanders • Based on work done in the STEVIN-programme • (as a severe test for interoperability)

  10. Documentation • Is always necessary, so hardly any additional effort • Partly in natural language • Partly formalized • Described under a particular formally identifiable attribute • With an explicit type for the value of the attribute • Possibly with further restrictions on the values (patterns, finite lists of values, constraints, etc.) • Represented formally and unambiguously • Any piece of documentation that can be formalized must be formalized, and must be put in the resource description (metadata of the resource)

  11. Documentation • Resource Descriptions • Component-based MetaData Infrastructure (CMDI) • One can define resource profiles as collections of components (which can contain components). • Many generally useable components are available • Resource profiles for most common resources are available • Component-based  flexibility • Flexibility: danger: diversity, no interoperability • Controlled by semantic interoperability (see below) • Not yet available but needed: profile(s) for tools • Supported by tools • Component and profile editors • Component and profile registries • Metadata editor

  12. Visibility • Each resource and its resource description must be stored at a CLARIN-centre • CLARIN-centres make resource descriptions available for metadata harvesting (using OAI-PMH) • Via harvesting the metadata, the metadata become available in the CLARIN resource catalogue • browsing via the Virtual Language Observatory (VLO) using faceted browsing • Search via a search interface (under development) • In the metadata and in the data • String search and structured search • Results if desired collected in a Virtual Collection

  13. Referability • By name or title is not sufficient • All the problems that natural language poses for communication: • not always unique (ambiguity) • language-specific Corpus Gesproken Nederlands • Variants in other languages: Spoken Dutch Corpus • limited knowledge of the foreign language  variants: Corpus Spoken Dutch, Dutch Spoken Corpus • Long, too redundant, • abbreviations/acronyms: CGN • Invites for errors • Spoken Dutch Cropus, Spken Dutch Corpus • URLs • Still too long/redundant (unless one uses shortened URLs) • Unstable, volatile • Persistent Identifiers (PIDs) are needed

  14. Referability • PIDs • Each CLARIN-Centre • must assign a PID to each resource (and/or to subresources) • Keep the PID resolution registry up-to-date • PID systems • Handle (preferred) • URN • Perhaps others (e.g. DOI)

  15. Accessibility • CLARIN infrastructure • Accessible at any time and from any place • IPR • CLARIN-NL promotes maximal open access of resources • is working on plans to implement policies and functionality to properly handle IPR and ethical restrictions • Researchers’ Mindset • Many researchers in the humanities are hesitant or even unwilling to share their resources with others • How to resolve this? With a carrot and a stick • CLARIN must accommodate reasonable wishes • CLARIN must prove benefits for researchers who put their resources there • Funding agencies must oblige researchers to do so (partially already so)

  16. Long Term Preservation • Necessary to make sure the resources can be shared with future researchers (that may be the producer!) • Each CLARIN-Centre is obliged to ensure long term preservation • Usually outsources to specialized centres • MI outsources to DANS • MPI outsources to internal Max Planck Gesellschaft organisation

  17. Interoperability • Interoperability of resources is the ability of resources to seamlessly work together • No manual ad-hoc adaptations • Adaptations occur automatically `behind the screens’ • Need for interoperability is high • Humanities researchers: not the required technical background • Interoperability • Syntactic interoperability and Semantic interoperability • Each subproject must try to achieve interoperability • Report any problems and make suggestions for adaptations • So that the resources are adapted to the infrastructure (in some cases) and vice-versa (in other cases) • Not easy, but the only way to get further is to actually try this and learn from it.

  18. Syntactic Interoperability • the formats of data are selected from a limited set of (de facto) standards or best practices supported by CLARIN • software tools and applications take input and yield output in these formats

  19. Semantic Interoperability • Focus on the semantics of Data Categories (DCs) • a privileged data category registry (DCR) is set up containing DCs: • unique persistent identifiers for DCs (PIDs), • their semantics, • a definition, • Examples • lexicalizations in various languages. • Each resource specific DC mapped to DC from the privileged DCR. • every researcher can use his/her own DCs • different DCs from different resources can be interpreted as identical in meaning, via the DC of the privileged DCR • In CLARIN-NL multiple (complementary) privileged DCRs are allowed. The primary is ISOCAT

  20. Semantic Interoperability • Achieving semantic interoperability is very hard • Many DCs are almost identical (principled/pragmatic/arbitrary reasons) • Some DCs in ISOCAT are not defined clearly • There are many similar DCs in ISOCAT • Relevant DCs are not easy to find in ISOCAT • Three actions taken • Held several workshops to discuss problems • Appointed a coordinator to deal with problems • Decided to implement RELCAT registry to specify relations between DCs

  21. Conclusions • CLARIN-NL requires systematic sharing of resources • Therefore requires researchers to work on • Documentation • Visibility • Referability • Accessibility • Long Term Preservation • Interoperability Of resources • For certain aspects this is relatively easy but it must be done • For other aspects this is very hard but it must be done so that we can learn • The approach described here may be a model for other countries working on the CLARIN-infrastructure • It may be a model for other resource sharing facilities (e.g. META-SHARE)

  22. Thanks for your attention!

More Related