1 / 37

Cooking with Sloan: Special DR6 Recipes

Cooking with Sloan: Special DR6 Recipes. Jordan Raddick, Ani Thakar Alex Szalay, Maria Nieto-Santisteban, Nolan Li, Wil O’Mullane, Adrian Pope, Tamas Budavari, George Fekete, Sam Carliles JHU Brian Yanny, Svetlana Lebedeva FermiLab Jim Gray Microsoft Research. Outline. SDSS-II and DR6

adli
Download Presentation

Cooking with Sloan: Special DR6 Recipes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cooking with Sloan:Special DR6 Recipes Jordan Raddick, Ani Thakar Alex Szalay, Maria Nieto-Santisteban, Nolan Li, Wil O’Mullane, Adrian Pope, Tamas Budavari, George Fekete, Sam Carliles JHU Brian Yanny, Svetlana Lebedeva FermiLab Jim Gray Microsoft Research

  2. Outline • SDSS-II and DR6 • CAS Data Access • SkyServer, ImgCutout, CasJobs • Help resources and sample queries • Restricted collab access • SDSS and other datasets • VO services • EPO content SDSS catalog data demo, March 30, 2007

  3. SDSS-II • Legacy • Continuation of SDSS-I (fill out 10k sq.deg.) • Completeness is same as for SDSS-I • Flux limits are the same • Target all galaxies with r_petro < 17.77, plus LRGs • SEGUE • Detailed 3-d map of the Galaxy • Spectra of 240,000 stars in the disk and spheroid • Age, composition and phase space distribution • CAS component cataloged in SegueDRx DB • Supernova Survey • Repeated scans of SDSS Southern Stripe over 3 mths/yr • Data not available in CAS yet, will be on DAS soon SDSS catalog data demo, March 30, 2007

  4. SDSS Data Overview • Catalog Archive Server (CAS) • Science parameters extracted to catalogs • Stuffed into relational DBMS (SQL Server) • Heavily indexed, optimized • Online access via SkyServer • Several levels of access, query tools Data Archive Server (DAS) FITS files (raw data) Images, spectra, corrected frames, atlas images, binned images, masks Online form-based access Rsync and wget file retrieval SDSS Data Release cas.sdss.org/collab/ skyserver.sdss.org www.sdss.org das.sdss.org/DRx-cgi-bin/DAS SDSS catalog data demo, March 30, 2007

  5. CAS Datasets (databases) • BestDRx • Latest, greatest calibration of the data • Photometric and spectroscopic objects • The default and most accessed (by far) dataset • TargDRx • The calibration from which spectroscopic targets were chosen • RUNS • All the runs (processings) other than Best, Target • SegueDRx • New with DR6 (SDSS-II) SDSS catalog data demo, March 30, 2007

  6. CAS Data Access • SkyServer: http://cas.sdss.org/collabdr6 • Also pwd access at http://cas.sdss.org/collabdr6pw for non-collab IPs • See message sdss-archive/2935 for uname/pwd • CasJobs • DR6 and DR6QA targets point to BestDR6 DB • TARGDR6, SEGUEDR6 targets • Need to have “collab” privilege set in user profile SDSS catalog data demo, March 30, 2007

  7. CAS Data Access • SkyServer • Synchronous access – need only web browser • Meant to support several levels of users • Visual tools to browse image and catalog data • API access, e.g. emacs interface, sqlcl (command-line) • Strict limits on execution time and output size(but higher for collaboration) SDSS catalog data demo, March 30, 2007

  8. Navigating SkyServer • Major sections linked from main page SDSS catalog data demo, March 30, 2007

  9. Finding your favorite object • Get position from SIMBAD SDSS catalog data demo, March 30, 2007

  10. Finding your favorite object • Advanced Tools -> Navigate • Enter RA/Dec in DMS or HH:MM:SS SDSS catalog data demo, March 30, 2007

  11. Finding your favorite object • Click to get position, magnitude • Notebook “shopping cart” SDSS catalog data demo, March 30, 2007

  12. Finding your favorite object • Explore – full SDSS data for object • Chart – create finding chart SDSS catalog data demo, March 30, 2007

  13. Searching the data • Search for data that meets certain criteria • Return matching data, and only matching data • Example: search for galaxies with spectra in Abell 168 • RA = 18.84, Dec = 0.26, radius = 3 arcmin • Limit to galaxies g < 18 • Return RA, Dec, magnitudes, redshift SDSS catalog data demo, March 30, 2007

  14. Searching the Data • Tools ->ImagingQuery • Get data asHTML or CSV • Constraintsby positionor imaging SDSS catalog data demo, March 30, 2007

  15. SQL • SQL – computer language for searching databases • Advantages • You get longer queries and more results returned • You can write more complex queries • You can use command-line tools SDSS catalog data demo, March 30, 2007

  16. SQL Concepts • Data are stored in a database • Similar data types are stored in tables • photoObj (photometry), specObj (spectroscopy), etc. • A small part of the photoObj table: SDSS catalog data demo, March 30, 2007

  17. SQL Concepts • Within a table: • Horizontal rows are individual data points, or records • Vertical columns are types of data, or columns • A request to a database to return data is called a query • Queries usually request data that meets certain constraints SDSS catalog data demo, March 30, 2007

  18. SQL as a foreign language • Languages have grammar and vocabulary • Dutch grammar • In sub-clause, verb goes at the end • English: • I think that astronomy is fun. • Dutch: • Ik denk dat het sterrenkunde leuk is. SDSS catalog data demo, March 30, 2007

  19. SQL Grammar • Select – choose which columns of data you want to see • From – choose the table(s) from which you want to retrieve data • Where – set constraints on the search SDSS catalog data demo, March 30, 2007

  20. Dutch vocabulary SDSS catalog data demo, March 30, 2007

  21. SQL vocabulary • Go to Help -> Schema Browser SDSS catalog data demo, March 30, 2007

  22. Learning SQL • Go to Help -> Sample SQL Queries • Go to Tools -> Search -> SQL • Copy, paste, modify SDSS catalog data demo, March 30, 2007

  23. Visual search results • Go to Advanced Tools ->Image List • Use query to fill form • Query must have:select x, ra, dec SDSS catalog data demo, March 30, 2007

  24. CasJobs • Go to Search Tools -> CasJobs • Best method for long or complex queries SDSS catalog data demo, March 30, 2007

  25. CasJobs • Register and log in • Quickmode: 1 minute • Submit: 24 hours • Results stored in MyDB SDSS catalog data demo, March 30, 2007

  26. CasJobs: other features • MyDB database to save results of your queries • Define your own functions, procedures too • Share your tables with collaborators (groups) • History of • Output data as FITS/CSV/VOTable • Plot data in browser • Table Import (upload) for your own data • Groups to share your results with collaborators • Command-line access Java tool also downloadable SDSS catalog data demo, March 30, 2007

  27. Collab-only access • collabdrx SkyServer sites • http://cas.sdss.org/collabdrx • http://cas.sdss.org/collabdrxpw • collab privilege in CasJobs • Restricted data, longer queues (DR6QA 24-hr queue) • If you have collab set, you will see these queues • If you don’t have it, email sdss-helpdesk@fnal.gov SDSS catalog data demo, March 30, 2007

  28. SkyServer help • Schema Browser • FAQ • Glossary – of SDSS terms • Algorithms • Helpdesk SDSS catalog data demo, March 30, 2007

  29. CasJobs help • Shares SQL Intro, Schema Browser with SkyServer • Has its own FAQ page • Lists differences between CasJobs and SkyServer due to distributed query execution • Advanced CasJobs Queries page • Neighbor searches with fixed and variable search radii • Cursors • Compound queries SDSS catalog data demo, March 30, 2007

  30. Find the right tool for the job • When you want to browse images, or when you know a position… • ToolsVisual Tools • Find images and data: Navigate • Browse objects one at a time: Explore • Find objects on a frame, compare your own observations: Finding Chart • View multiple objects or thumbnail search results: Image List SDSS catalog data demo, March 30, 2007

  31. Finding the right tool (contd.) • When you don’t know specific objects… • ToolsSearch • Find objects in a patch of sky: Radial or Rectangular • Simple search for imaging data: Imaging Query Form • Simple search for spectral data: Spectro Query form • More sophisticated searches:SQL • Long, complex queries: CasJobs SDSS catalog data demo, March 30, 2007

  32. Finding the right tool (contd.) • When you have a list of objects, and you want to see what SDSS knows about them… • ToolsObject Crossid • Imaging crossid: Upload • Spectro crossid: SpecList SDSS catalog data demo, March 30, 2007

  33. Data access: general tips • Use collab or astro sites • Less “frills”, more direct access to tools • Some extra features • e.g. Imaging/Spectro Form Query not on public page • Each release has separate sites • http://cas.sdss.org/collab/ (the public release) • http://cas.sdss.org/collabdr6/ (not yet public) • Use Contact link to contact help-desk (not direct E-mail) SDSS catalog data demo, March 30, 2007

  34. CAS Dos and Don’ts • Do not submit a query unless you have some idea how long it will take! • It could tie up the server for hours (sometimes days)! • Use “select count(*)” or “select top 10” instead • Do use predefined views for commonly-used objects and parameters • PhotoObj, PhotoPrimary for photometry • PhotoTag if you only need popular fields • SpecObj for spectra SDSS catalog data demo, March 30, 2007

  35. Educational Resources • Extensive education content in SkyServer • Use Projects link on the menu at the top • K-12 and college level student exercises and teacher resources SDSS catalog data demo, March 30, 2007

  36. Educational projects • Go to public site – http://cas.sdss.org • Go to Projects • Great as labs for intro classes SDSS catalog data demo, March 30, 2007

  37. Focused practice • Get into groups of two • Share a computer • Consider some task you do often in research or teaching • Some ideas in your handout • Use SkyServer or CasJobs to accomplish the task • We will answer questions, offer advice • You have 45 minutes… go! SDSS catalog data demo, March 30, 2007

More Related