1 / 17

DESY Site Report: Overview of Ongoing Activities and Projects

This report provides an overview of ongoing activities and projects at DESY, including storage and data management, dCache, ExaStore, user registry project, and Windows migration project.

ckathryn
Download Presentation

DESY Site Report: Overview of Ongoing Activities and Projects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DESY Site ReportHEPiX/HEPNTFermilab2002-10-23Knut Woller

  2. Overview I will focus on ongoing activities and projects: • Storage and data management • dCache • ExaStore • User Registry Project • Windows Migration Project • Mail Consolidation DESY Site Report

  3. Storage and Data Management New requirements and challenges: • Need to decrease storage costs • Increasing number of clients burdens HSM • Distributed clients create awkward data paths, and distributed NFS does not scale • The “Traveling Scientist” requires mobility • Users are increasingly unable or unwilling to judge features or cost of a specific store. They just want to use it. DESY Site Report

  4. About dCache • Distributed cache between clients and HSM • Collaborative development at DESY & FNAL • In production use at DESY and FNAL • More labs are looking into it • DESY currently runs about 30TB read pool on IDE RAID servers • All major DESY groups use it by now • For us, it is the method to access HSM data DESY Site Report

  5. dCache Features • Allows the use of cheap tape media by largely reducing the number of mounts • Coordinates the site wide data staging and reduces data management manpower • Supports several HSMs (OSM, EnStore, Eurogate) • Can be transparently used by applications through C-API (ROOT supports dCache) • Scales well to thousands of clients and hundreds of pool servers • Can be used in GRIDs (bbFTP, gridFTP) DESY Site Report

  6. dCache Development • DESY / FNAL project is well advanced • Presentations have been made at recent HEPiX and CHEP conferences • Project information is on http://www-dcache.desy.de • We plan to set up a central read disk pool of 100+TB when we migrate to large, cheap tapes (STK 9940B) in a few months. DESY Site Report

  7. ExaStore • Since 1999, major user groups have demanded a “Large Central File Store” at DESY • Features: Multi-Terabyte, high performance, single filesystem view, random access • AFS will not scale to this size • dCache does not fit the requirements • Commercial NAS solutions do not scale well • EXANET came along in 2000 with a product proposal that suits our needs DESY Site Report

  8. About ExaStore • Seen from the outside, the ExaStore is a highly scalable, high performance NAS (or a huge virtual disk) • Internally, it is built from disk and CPU servers and independent RAID arrays. ExaStore’s spice is • The use of commodity components • Their cluster file system • Their redundant server mesh • ExaStore scales in (at least) two dimensions: • In capacity by adding disks • In performance by adding nodes and/or uplinks DESY Site Report

  9. Why Exastore at DESY? • Because the current jungle of cross mounted NFS disks is an administrative nightmare • Because NFS data management at DESY today is handled decentrally in the user groups. IT wants to fill this gap to make better resource use. • Because scaling the current system of distributed NFS servers reduces stability and manageability • Because current NAS solutions are limited to 12-18TB per box and a fixed number of uplinks and server nodes • Because we do not think it would be wise to invent our own SAN/NAS solution. DESY Site Report

  10. ExaStore Experiences • First test system at DESY since April, in beta test since June (4 nodes, 1.5TB) • No crashes in four months • Performance is not yet where we want it to be, but well on the road • We want to acquire a production system with 8 nodes and 12 TB (management approval pending) DESY Site Report

  11. User Registry Project • DESY User Registry is old, limited, inflexible • Number of user groups is increasing • Each new complex software system today comes with a proprietary registry (e.g. mailserver, calendar server, Oracle, SAP, …) • Interfaces to HR database, phonebook etc. are required • We need a site wide metadirectory toolbox • Groups have a large demand for delegations of rights DESY Site Report

  12. Project Approach • Design phase started in January • We have a clear functional description now • We looked in to commercial (Tivoli, CA, …) and open source (Ganymede) tools, none of which seem to fit our needs • We are gathering troops to start coding • Platform account (unix, windows, kerberos) should be manageable in Q2/2003 • Platform adaptors will take some time DESY Site Report

  13. Windows Migration Project • The DESY Windows Domain is still NT4 • We started rolling out W2K and WXP clients in the DESYNT domain (mostly notebooks) • Basic software support (netinstall) for WXP desktops in DESYNT available this year • Domain servers are NT4, newer ones W2K • .net server look promising, but are not in production use yet • Where possible, we are skipping W2K clients DESY Site Report

  14. W2K Migration Status • New project team has been formed within IT • We are finalizing the site wide AD design • New hardware has been / is being acquired • Homedir storage is under reconsideration • We plan to have a working domain in Q1/2003 • Migration start foreseen in Q2/2003 • DESYNT will stay alive for control systems DESY Site Report

  15. Mail Consolidation • We are still in the sad state of supporting sendmail, Exchange, and PMDF • We experience load and capacity problems on all three systems • User ‘requirements’ (real or not) have limited us in the past years • Next step will be mail routing consolidation to get rid of PMDF • We want to end up with one mail router and one mail server solution, both yet unnamed DESY Site Report

  16. In General … • … we have been able to increase our IT staff with bright, young colleages (IT is back to 1999 staffing level) • … we start seeing synergy effects by treating windows and unix systems in one group (e.g. Samba, hardware standards) • … we have been able to start a few major efforts and projects • … we are striving for more coherence between Hamburg and Zeuthen • … much of our effort is still required to clean up or legacy from the past (technologically and socially) • … I think we have a few very well working and scalable solutions, e.g. in mass storage (dcache), Linux support, printing DESY Site Report

  17. That’s It Thank you for your attention DESY Site Report

More Related