Cds 2013 accomplishments and 2014 directions
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

CDS 2013 Accomplishments and 2014 Directions PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

CDS 2013 Accomplishments and 2014 Directions. John Allison (for Greg Stossmeister ), Steve Williams, Jody Williams, Charlie Martin, Mike Daniels. Collaborative Tools and Metadata. CTM Accomplishments 2013. Field Catalog 2.0 Deployed – May 2013 - Catalog Maps with A/C Cameras & Replay

Download Presentation

CDS 2013 Accomplishments and 2014 Directions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cds 2013 accomplishments and 2014 directions

CDS 2013 Accomplishments and 2014 Directions

John Allison (for Greg Stossmeister), Steve Williams, Jody Williams, Charlie Martin, Mike Daniels


Cds 2013 accomplishments and 2014 directions

Collaborative Tools and Metadata


Cds 2013 accomplishments and 2014 directions

CTM Accomplishments 2013

  • Field Catalog 2.0 Deployed – May 2013

  • - Catalog Maps with A/C Cameras & Replay

  • - Catalog Theming with Responsive Design

  • - Nagios for monitoring FC servers and product streams

  • - Catalog Ingest implemented with metadata vetting

  • - New workflows for Catalog Admins

  • First Campaigns:MPEX

    • SAS

  • User Survey implemented following the Field Campaigns

  • - Guided redesign of field catalog navigation


  • Cds 2013 accomplishments and 2014 directions

    CTM Accomplishments 2013

    • Field Catalog 2.1 Deployed – November 2013

    • - Re-designed Navigation

    • - Functional Mockups User Feedback

    • - Reports (Db based)

    • - Multi-author facility status

    • - Catalog Ingest debugged – John & Tom

    • - Incorporating DOW radar data – Catalog Maps

    • - Dedicated MySQL server

    • - Remote Field Catalog Server

    • Field Campaigns:OWLeS

    • CONTRAST & HIWC (in January)


    Cds 2013 accomplishments and 2014 directions

    CTM Accomplishments 2013

    • Field Projects:

    • - HIWC Forecasting Exercise

    • - MPEX

    • - SAS

    • - DEEPWAVE Dry Run

    • - OWLeS

    • Presentations

    • - Erik: SSG – Browser-side Field Catalog 2.0

    • - Erik: ESIP – NCAR EOL’s Experience using Twitter Bootstrap and Responsive Web Design for Field Catalog 2.0

    • Storage Appliance Task Force - Erik


    Cds 2013 accomplishments and 2014 directions

    CTM Accomplishments 2013

    Data archive

    - Continued metadata maintenance and user support

    - Delivered 3.7 TB online data to 1458 unique users

    Codiac improvements

    - inline video previews

    - station-based static image previews

    - updated EOL branding for data.eol.ucar.edu

    Continued work with DMG on ACADIS and metadata issues


    Cds 2013 accomplishments and 2014 directions

    CTM 2014 Plans

    Deploy Field Catalog 2.1.1, 2.2

    - Refactoring from OWLeS, CONTRAST, HIWC

    - Model products in Catalog Maps

    - IRC Bot replacement – Chris B.

    - Web Mapping Service (WMS)

    - Flight simulation - Dry Runs/ISF products – Charlie

    - Rapid updates of aircraft info – Catalog Maps

    - Metrics

    - kml text

    Merging of Master Lists with EMDAC DB


    Cds 2013 accomplishments and 2014 directions

    Data Management Group


    Cds 2013 accomplishments and 2014 directions

    HIGHLIGHTS OF GENERAL DMG ACCOMPLISHMENTS - 2013

    • Provided feasibility and data management planning support to 10+ future Projects

    • Worked on 18+ projects during 2013 in various stages of completion and support

    • (data collection, processing, quality assurance, archival, and dissemination)

    • Major Projects in 2013 included SAS/NOMADSS/SOAS, MPEX, DC3, and DYNAMO

    • Major Upper Air “Composite” Dataset created for DYNAMO ( ~ 24,000 Soundings)

    • Completed extensive HIPPO archive (EOL HIPPO team won the UCAR Scientific and

    • Technical Advancement Award)

    • - Completed the Plone to Drupal web content transition

    • - Completed processing of archived NWS high vertical resolution Soundings (> 750,000)

    • Provided Support for both EOL and NCAR Strategic Plans

    • Continued membership on both internal and external committees (EOL Web advisory,

    • EOL Internal Advisory, EOL Diversity Committee, NCAR Strategic Planning, SEA,

    • NCAR Allocation Review Panel, GRUAN … )

    • Mentored and supported 4 UCAR Interns (DCERC, TIP, High School) and students

    • Attended Training Classes and Professional Conferences (presentations/posters)

    • Participation in UCAR/NCAR Data Citation and IT Strategic Planning Groups

    • Collaboration with CISL on EOL data off-site backup and future on-line data

    • JTECH journal article accepted for publication on DYNAMO Sounding quality control


    Cds 2013 accomplishments and 2014 directions

    EOL DATA ARCHIVE STATUS

    EOL has identified over 400 Field Projects in the past 45 years that we have participated in or have data for in our archives. The Line Islands Experiment (LIE) was the first major NCAR Field Project in 1967

    EOL Data Legacy Archives

    (Inventory as of Nov 2013)

    EOL Digital Archives

    Data housed on NCAR HPSS and EOL Servers:

    ~ 6000 datasets

    > 17 million files

    > 150 TB data

    Since 1995:

    11,553 users placed > 90K orders, receiving 15 TB data online

    516 users requested tapes totaling 4,492 GB


    Cds 2013 accomplishments and 2014 directions

    EOL DATA LEGACY AND STEWARDSHIP ACTIVITIES - 2013

    • Continued Inventory of EOL Archives (Tape, HPSS, Hardcopy, other misc. media ….)

    • VHS Aircraft Tape digitization nearing completion ( ~ 900 tapes digitized over 4 years)

    • Development of Legacy Project web pages (provide access to descriptions, data,

    • publications, documents, photography ….. )

    • Migration of Project web pages to Drupal from older html format

    • Creating EMDAC metadata for existing digital data as located

    • Collaboration with NCAR Library/Archives on scanning documentation

    • Performing EOL-wide Instrumentation and Data Audit

    • Revising EOL Data Policy in coordination with NSF

    • Evaluation and testing of additional media migration

    • Article on EOL Legacy/Stewardship Data Activities (UCAR AtmosNews, Oct 2013)

    • Project relies on strong involvement of students and DCERC interns

    • 20 Countries

    • 40 research ships

    • 12 research aircraft

    • > 1000 GATE papers


    Cds 2013 accomplishments and 2014 directions

    HIGHLIGHTS OF DMG ARCTIC ACCOMPLISHMENTS - 2013

    • Participated in ACADIS Project (Project Management, Working SubGroups, Special

    • Requirements Data, ADAC, NSF Panel Review)

    • Cleaning up of Arctic metadata (2194 Data Sets) and developing data citations

    • Providing review of ACADIS organization and management structure

    • Completed USCGC Icebreaker Polar Star Mapserver/Ice Observations Project

    • Completed PacMARSMapserver/Archive (includes ~ 150 data layers and village profiles)

    • Provided Planning Support to the Distributed Biological Observatory (DBO) Project

    • Participated in bi-annual Arctic Observing Summit (Vancouver)

    • Attended Project meetings and Professional Conferences (presentations/posters)


    Cds 2013 accomplishments and 2014 directions

    DMG 2014 PLANS

    • DOI generation for all datasets

    • Investigate the use of another DCERC student

    • Convert the Master List tool to the EMDAC database

    • Expansion of arctic metadata entry tool for EOL-wide holdings

    • Data Legacy Report for EOL Management

    • Data Management support for HIWC, CONTRAST and DEEPWAVE

    • Pursue additional funding for Polar projects


    Cds 2013 accomplishments and 2014 directions

    Systems Infrastructure Group


    2013 accomplishments systems infrastructure

    2013 Accomplishments:Systems Infrastructure

    • Security – no security incidents; EOL led UCAR’s diminishing phishing attempts. Webmail was put back outside the perimeter.

    • Reliability: no major server outages except the SCALE issues

    • Drupal web server

    • Hyper-V cluster – virtualized many aging servers (eol-rt-data, eol-epicac, wiki, weather,…)


    Sig field work

    SIG Field Work

    ops-in-a-box, MPEX, SAS (SOAS, NOMADDS)

    working with CTM for OWLES (setting up a new server, ftp areas)


    Systems infrastructure

    Systems Infrastructure

    • Storage issues – SCALE issues, worked a lot on debugging, working with Contracts, the reseller, and VP of SCALE. Tier 1 storage task force – purchase NetApp

    • Windows XP to Windows 7, RHEL to Scientific Linux

    • New Servers: eol-doppel, eol-doppel2, barolo, aeolus (CTM), farskol (CTM), wombat (CTM), eol-apar (RSF), vdras (RSF), NetApp

    • Decommissioned Servers: bora, m5, malbec, eve, SCALE

    • Committees: CSAC, NCAB, ACCISS, CFM, SINAP, EWAG, IAC, ITC, ITC Summit, Strategic plan work for UCAR/NCAR

    • Special Projects – RSF and Front, Earthcube real-time data workshop support, RAF remodel to include on-site Ops Center.


    Sig jira requests

    SIG jira requests


    Systems infrastructure future plans

    Systems Infrastructure – Future Plans

    • New “Fire Drill” procedures

    • NetApp integration

    • SINAP Survey Round II

    • Google Apps

    • Equipment move to Mesa Lab computer room

    • Field Programs: HIWC, CONTRAST, DEEPWAVE


    Cds 2013 accomplishments and 2014 directions

    Software Systems Group


    2013 accomplishments software systems

    2013 Accomplishments: Software Systems

    Field Programs


    2013 accomplishments software systems1

    2013 Accomplishments: Software Systems

    Hardware Systems

    • HIAPER Cloud Radar (HCR)

    • SPOL

    • 3V-CPI

    • GV AVAPS

    • 449 Profiler

    • Microwave Temperature Profiler (MTP)

    • GV Satcom

    • CSWR Doppler on Wheels (DOW)

    • Laser Air Motion Systems (LAMS)

    • GV & C-130 Data Systems

    • ISFS


    2013 accomplishments software systems2

    2013 Accomplishments: Software Systems

    Software Package Developments


    2013 accomplishments software systems3

    2013 Accomplishments: Software Systems

    Other Activities

    • EOL Development Conference

    • EOL Software Summit

    • Summer Intern: Mike Patterson (HTML5)

    • SEA Conference

    • EOL Software Center (Drupal)

    • EarthCube Real-Time Workshop

    • AMS Radar Conference

    • RIC security audit

    • A-10 “Ice Breaker”


    Looking forward to 2014 software systems

    Looking Forward to 2014: Software Systems

    • IFRACS

    • CONTRAST

    • DEEPWAVE

    • ISF: The “Year of the Sounding”

    • APAR

    • SSG Student Assistant

    • Field Catalog cross-dev w/ CTM


    Cds 2013 accomplishments and 2014 directions

    Final Remarks


    Cds 2013 accomplishments and 2014 directions

    Overall CDS Funding (FY06-FY14)

    NOTE: For FY14, NSF Spec and Non-NSF amounts are projections, only.


    Cds 2013 accomplishments and 2014 directions

    Overall CDS Spending (FY06-FY14)

    NOTE: For FY14, NSF Spec and Non-NSF/Other amounts are projections, only.


    Cds future directions in 2014

    CDS Future Directions in 2014

    • Pursue outside funding(!): EarthCube, Polar, Data Legacy, Software Infrastructure, other EOL-related

    • Creation of an Open Source software environment for our community

    • Stronger lab-wide integration of software developments and data services activities

    • Plans for longer-term enhancements of software and data services

    • Emphasis on Project Management


  • Login