Status of apogee data products reduction and analysis
Download
1 / 36

STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS - PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on

STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS. Jon Holtzman (NMSU) David Nidever (UVa) Ana Garcia-Perez (UVa) Carlos Allende-Prieto (IAC) Szabolcs Mezaros (IAC) The APOGEE team. Data Overview. Target selection : input photometry, selection flags, extinction, etc.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS' - tariq


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Status of apogee data products reduction and analysis

STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS

Jon Holtzman (NMSU)

David Nidever (UVa)

Ana Garcia-Perez (UVa)

Carlos Allende-Prieto (IAC)

Szabolcs Mezaros (IAC)

The APOGEE team


Status of apogee data products reduction and analysis

Data Overview

  • Target selection : input photometry, selection flags, extinction, etc.

  • Observations: up-the-ramp readouts + telescope/weather information  raw APOGEE data files. Exposure sets are taken in visits: multiple frames at different dither positions

  • Data reduction:

    • Visit analysis: basic calibration, extraction, sky correction, initial RV

    • Visit combination: wavelength resampling, refined RVs

  • Chemical analysis (ASPCAP)

    • Stellar parameter determination

    • Individual element abundances

2


Status of apogee data products reduction and analysis

Summary data files

Most major results summarized in two data files:

allVisit : merges target information, individual visit RVs, individual visit stellar parameters. One record per visit per star

allStar : merges target information, average RVs (soon, combined frame RV), average individual visit stellar parameters (soon, stellar parameters from combined frames)

Available in both FITS table, and (wide!) ASCII file

Files are sorted by RA, with accompanying RA “index” table (FITS table version) for faster searches

allStar file contains ID records (FITS table version) of individual visits in allVisit file


Status of apogee data products reduction and analysis

allVisit contents

rv_teff: effective temperature used for RV template

rv_logg: log g used for RV template

rv_feh: [Fe/H] used for RV template

rv_chi2: chi2 of RV fit

snr: median S/N ratio of spectrum

synthfile: synthetic spectrum file name

psfile: plot file

teff : ASPCAP Teff from visit

logg : ASPCAP logg from visit

vmicro : ASPCAP vmicro from visit

metals : ASPCAP metals from visit

c : ASPCAP stepar carbon from visit

n : ASPCAP stepar nitrogen from visit

alpha : ASPCAP alpha from visit

aspcap_chi2: ASPCAP chi2 from visit fit

aspcap_vers : ASPCAP version

ak : extinction used in target selection

akmethod : extinction method used

ak_wise_allsky : extinction using WISE ALLSKY release

wash_m : Washington M

wash_m_err

wash_t2 : Washington T2

wash_t2_err

ddo51 : DDO 51

ddo51_err

irac_3_6 : IRAC 3.6micron mag

irac_3_6_err

mag_4_5 : 4.5micron mag (IRAC or WISE)

mag_4_5_err

irac_5_8 : IRAC 5.8micron mag

irac_5_8_err

irac_8_0 : IRAC 8.0micron mag

objid : TMASS-style object name

file: apVisit file name

fiber: fiber number

plate: plate number

mjd: mjd observation date

location : field location ID

field: field name

ra : RA

dec : DEC

glon : galactic longitude

glat : galactic latitude

jmag : 2MASS J

hmag : 2MASS H

kmag : 2MASS K

primtarg : primary target flag

sectarg : secondary target flag

jd: JD of observations

bc: Barycentric correction

vtype: Radial velocity method type

vrel: derived radial velocity

vrelerr: error in radial velocity

vhelio: derived barycentric RV

vlsr: derived VLSR

vgsr: derived VGSR


Status of apogee data products reduction and analysis

allStar contents

obj : TMASS-style object name

locid : field location ID

field : field name

jmag : 2MASS J

hmag : 2MASS H

kmag : 2MASS K

ra : RA

dec : DEC

glon : galactic longitude

glat : galactic latitude

ak : extinction used in target selection

akmethod : extinction method used

ak_wise_allsky : extinction using WISE ALLSKY release

primtarg : primary target flag

sectarg : secondary target flag

nvisits : number of visits

pre : =1 for pre-2012 shutdown, otherwise 0

vavg: average RV

vscatter : scatter in RV

snr : S/N of combined frame

teff_avg : Average ASPCAP Teff from multiple visits

logg_avg : Average ASPCAP logg from multiple visits

vmicro_avg : Average ASPCAP vmicro from multiple visits

metals_avg : Average ASPCAP metals from multiple visits

c_avg : Average ASPCAP stepar carbon from multiple visits

n_avg : Average ASPCAP stepar nitrogen from multiple visits

alpha_avg : Average ASPCAP alpha from multiple visits :

teff_sig : Sigma of ASPCAP Teff from multiple visits

logg_sig : Sigma of ASPCAP logg from multiple visits

vmicro_sig : Sigma of ASPCAP vmicro from multiple visits

metals_sig : Sigma of ASPCAP metals from multiple visits

c_sig : Sigma of ASPCAP stepar carbon from multiple visits

n_sig : Sigma of ASPCAP stepar nitrogen from multiple visits

alpha_sig : Sigma of ASPCAP alpha from multiple visits

teff : ASPCAP Teff from combined frame

logg : ASPCAP logg from combined frame

vmicro : ASPCAP vmicro from combined frame metals : ASPCAP metals from combined frame

c : ASPCAP stepar carbon from combined frame

n : ASPCAP stepar nitrogren from combined frame alpha : ASPCAP alpha from combined frame aspcap_chi2 :ASPCAP chi2

aspcap_vers : ASPCAP version

wash_m : Washington M

wash_m_err wash_

t2 : Washington T2

wash_t2_err

ddo51 : DDO 51 ddo51_err

irac_3_6 : IRAC 3.6micron mag

irac_3_6_err

mag_4_5 : 4.5micron mag (IRAC or WISE) mag_4_5_err irac_5_8 : IRAC 5.8micron mag irac_5_8_err

irac_8_0 : IRAC 8.0micrton mag, irac_8_0_err


Status of apogee data products reduction and analysis

Other data products

  • Data model available at http://data.sdss3.org/datamodel, data at http:/data.sdss3.org/sas/bosswork/groups/apogee (to be renamed…)

  • Raw data: apR-[abc]-ID8.fits (in APOGEE_DATA/)

  • Reduced data (in APOGEE_REDUX/vers/)

    • Basic calibrations: red/MJD5/ap2D-[abc]-ID8.fits

    • Extracted 1D: red/MJD5/ap1D-[abc]-ID8.fits

    • Sky corrected: plates/PLATE4/MJD5/apCframe-[abc]-ID8.fits

    • Dither combined: plates/PLATE4/MJD5/apPlate-[abc]-ID8.fits, or separated apVisit-PLATE4-MJD5-FIBER3.fits (with initial RVs in headers), native wavelength scale (with wavelength array)

    • Visit combined: stars/LOC4/apStar-NAME.fits: rebinned to fixed log lambda dispersion

  • Chemical analysis (in APOGEE_ASPCAP/vers/)

    • Best fit stellar parameters and model spectra from individual visit spectra in results/aspcap-PLATE4-MJD5.fits

6


Status of apogee data products reduction and analysis

Sample spectra

Spectra plots available on embedded web pages, + RV plots, links to FITS files

7


Status of apogee data products reduction and analysis

DR10

  • DR10 contents still TBD, but will likely include

  • Table of all possible targets with target selection data

  • Design, field, plate data

  • allVisit table

  • allStar table

  • Separate ASPCAP tables for different versions? To include …?

8


Status of apogee data products reduction and analysis

Data quantity status

  • Sky coverage

  • S/N and number of visits

  • Reduction processing: visit processing routine and current, happens within 2-3 days of data taking; combination processing happening ~monthly

  • ASPCAP processing: uniform (but non-optimal) processing through ~March 2012

  • S/N vs H…

9


Status of apogee data products reduction and analysis

Data quality: reduction

Mostly relatively standard reduction steps

Basic calibration (flats, darks, linearity):

flats seem stable, minor improvements have been made over current pipeline version

Darks need some analysis

Linearity corrections not currently implemented

Extraction

Currently done with empirical PSF, considering 3 fibers at a time

Sky correction: telluric and sky subtraction

Dither combination

Visit RV

Visit combination / refined RVs



Status of apogee data products reduction and analysis

Data quality: RVs

Velocity Uncertainty

Scatter from multiple measurements

Some real variability

Peak RV scatter ~100 m/s

Velocity zeropoint

Comparison to GC RVs

RV offset average -0.26 +/- 0.2 km/s


Status of apogee data products reduction and analysis

Data quality: ASPCAP

ASPCAP tuning/development still active

Tuning of linelists

Fit method (PCA vs flux space) being changed

Parameter space (7 param vs 6 param plus fixed/coupled microturbulence) still under consideration

Fitting of combined spectra as well as individual visit spectra

Some tests of ASPCAP stellar parameters available

Repeatability: internal precision

Clusters

Kepler asteroseismic targets


Status of apogee data products reduction and analysis

ASPCAP precision

  • Sample:

  • Combined S/N>220

  • N(ASPCAP>=5)


Status of apogee data products reduction and analysis

ASPCAP accuracy: metallicities

  • ASPCAP metallicities seem fairly good near solar, but are systematically high for metal-poor clusters

  • TBD: understanding and decision on whether to implement calibration relations


Status of apogee data products reduction and analysis

ASPCAP accuracy: gravities

Comparison with asteroseismic gravities

  • ASPCAP gravities systematically high:

    • ~0.4 dex near solar

    • ~1 dex for metal-poor clusters

  • Significant issue for distances

  • Again, calibratable or need to solve?


Status of apogee data products reduction and analysis

Data quality: Extinctions

Catalog RJCE extinctions derived assuming fixed color for all stars: expected metallicity dependence could give systematic errors of 0.1-0.2 mag in A(K)

Simple to implement metallicity dependent A(K)

Still some uncertainties at metal-poor end: extinctions derived from spectroscopic temperature and color-temp relation don’t quite match RJCE extinctions


Status of apogee data products reduction and analysis

Conclusions

APOGEE data is flowing through reduction pipeline

Much processed through ASPCAP, soon to be routine

Data products available in SAS

Spectra

summary tables

ASPCAP fits

Important to be aware of known issues: publication quality TBD?

We’re interested to hear about unknown issues!


Status of apogee data products reduction and analysis

NMSU 1m – APOGEE Fiber feed

Hope to implement late summer/fall

Design work by Nick MacDonald + spare fibers: target + sky fibers

Main goals:

Calibration star targets

RV monitoring

Bright star survey

Challenge: robotic spectroscopy – acquisition and guiding


Status of apogee data products reduction and analysis

Preliminary distances and abundance gradients with APOGEE : Michael Hayden, Jon Holtzman (NMSU)

Determine mean M_H of PDF given Te, log g, [Fe/H] and adopted errors, with prior on IMF and SFH; use H and WISE reddenings to get dist (similar to Santiago et al SEGUE distances)

Coming: priors on spatial distribution, selection function, better error treatment (e.g., Burnett and Binney)

  • Inner disk coverage with ASPCAP still limited but should be available soon

  • Initial gradient results comparable with SEGUE (Cheng et al.)


Status of apogee data products reduction and analysis

Line Widths/ Resolution (1) Michael Hayden, Jon Holtzman (NMSU)

ThAr

Solid: fiber 150

Open: fibers 10/290

Each point is median of results from lots of frames (few per night for lamps, one per night for skies)

Simple Gaussian fits

Some lines appear to be resolved (blends)

UNe

Sky

  • Wavelength range/dispersion as expected

  • Resolution close to or achieves SRD specification of R~22500 (significant improvement from pre-shutdown in red)

22

Dec 8, 2011


Status of apogee data products reduction and analysis

Line Widths/ Resolution (1) Michael Hayden, Jon Holtzman (NMSU)

ThAr

Solid: fiber 150

Open: fibers 10/290

Each point is median of results from lots of frames (few per night for lamps, one per night for skies)

Simple Gaussian fits

Some lines appear to be resolved (blends)

UNe

Sky

  • Wavelength range/dispersion as expected

  • Resolution close to or achieves SRD specification of R~22500 (significant improvement from pre-shutdown in red)

23

Dec 8, 2011


Status of apogee data products reduction and analysis

Line Widths/ Resolution (2) Michael Hayden, Jon Holtzman (NMSU)

  • Stability and monitoring: line widths appear to be constant in time (mostly?)

  • Significant variation across chip not unexpected

24

Dec 8, 2011


Status of apogee data products reduction and analysis

Line Widths/ Resolution (3) Michael Hayden, Jon Holtzman (NMSU)

  • Issues

  • Moderate undersampling in blue

  • Resolution varies across chip (not always best in center!)

  • Beware sky lines for LSF characterization?

  • Caveat:

  • Measurements from uniformly illuminated fibers

  • Possible LSF measurements from fits to telluric lines, or perhaps asteroid (or solar analog)

25

Dec 8, 2011


Status of apogee data products reduction and analysis

S/N and throughput Michael Hayden, Jon Holtzman (NMSU)

  • SRD calls for S/N=100 at H=12.2 in 3 hr exposure per pixel at 1.6 microns, airmass=2, seeing=1.5”, clear conditions. Pixel is taken to mean a “Nyquist sampled pixel”; since dithering provides spectra at ~2X Nyquist sampling, this means S/N=70.7 per dithered pixel.

  • S/N calculated for all 500s exposures taken to date, using median S/N for 12<H<12.2, where noise comes from noise model

  • Some S/N verification from looking at “featureless” region of tellurics; ratio of observed/expected scatter generally between 1.0 and 2.0

  • Dotted lines are SRD requirement assuming signal-limited exposures

Note that S/N in excess of SRD doesn’t necessarily imply we are overexposing because we could reduce number of exposures

26

Dec 8, 2011


Status of apogee data products reduction and analysis

Persistence Michael Hayden, Jon Holtzman (NMSU)

  • APOGEE detectors have noticeable persistence issues

  • Blue detector has “superpersistence” over top ~1/3rd

  • Some characterization done in spring, but little or no effort since then

  • Attempting to minimize as much as possible:

  • Cold shutter installed and in routine use

  • Dome flats are only nighttime frames taken that aren’t science frames; taken AFTER science exposures of a plate, followed by two short dark

  • Evening cals minimized, and taken early, with more complete calibration set taken in morning; kept cal exposures to a “minimum”

27

Dec 8, 2011


Status of apogee data products reduction and analysis

Dither pair accuracy Michael Hayden, Jon Holtzman (NMSU)

  • Modifications made to mechanism algorithm between runs 2 and 3 made significant improvements!

  • SRD spec: 0.5 micron accuracy or 0.028 pixels (dotted lines)

  • Algorithm for combining dither frames should handle arbitrary shift size so long as it is well measured, although deviations from 0.5 lead to correlated noise issues

  • Detailed tests on understanding whether 2 samples are sufficient for full LSF reconstruction have not yet been done (dither combination of monochromatic sources complicated by intensity variations): test on single line looks good

28

Dec 8, 2011


Status of apogee data products reduction and analysis

Observing software Michael Hayden, Jon Holtzman (NMSU)

  • Observing software functional and relatively stable:

  • ICS software STABLE : some modifications have been made for dithering and for safety interlock. Possible desire to implement additional exposure type keywords

  • Instrument control MOSTLY through SOP: few items still done with STUI scripts: cartridge change, evening cals, morning cals. Appears to be going smoothly, but possible desire to incorporate everything into SOP (priority? manpower?)

  • Python quicklook actor MOSTLY STABLE: handles header annotation of frames, checksum vs raw frames, initiates quicklook, quickred, bundling. Some issues with long term stability, so restarted every day automatically (TBD priority for modification? Manpower?)

  • Quicklook/webapp for observing support to be discussed later

  • Morning cron job checks for missing bundled frames, creates MD5 sums for SAS transfer

  • SAS transfer of raw data running stably during subsequent day

29

Dec 8, 2011


Status of apogee data products reduction and analysis

Observing support Michael Hayden, Jon Holtzman (NMSU)

  • Observers have been given primary contacts:

  • Frinchaboy: scheduling issues

  • Holtzman : actor / observing issues

  • Shetrone: observing issues

  • Alerts set up for instrument parameters, disk space

  • Things seem to be running fairly smoothly (from team perspective, but it’s really the observer perspective that is important here!)

30

Dec 8, 2011


Status of apogee data products reduction and analysis

Calibration data Michael Hayden, Jon Holtzman (NMSU)

  • Calibration data is taken on a regular basis:

  • Daily internal cals:

  • ThArNe and UNe lamps at 2 dither positions

  • Quartz exposure

  • 3 long darks

  • Internal flats

  • Monthly internal cals, plan not yet routine – needs improved procedure/responsible party

  • 10 long darks

  • 10 internal flats

  • Persistence monitor

  • Sparse pack quartz

  • On-telescope calibration data:

  • Mirror petal flats taken after every cartridge to 1) provide PSFs, 2) provide fiber to fiber throughput

  • Sky frames taken once per night (4 150s exposures) for LSF monitoring (but note possible issues with these)

  • Every plate has sky and telluric star fibers for sky subtraction and telluric correction

  • Overall approach has been relatively conservative, however, little or no impact on efficiency from internal cals, some from on-telescope cals

31

Dec 8, 2011


Status of apogee data products reduction and analysis

Calibration data (2) Michael Hayden, Jon Holtzman (NMSU)

  • Calibration data suggests instruments appears to be relatively stable:

  • Wavelength cals

  • Flat fields

  • Darks

  • LSFs

  • Appears that current internal calibration data is sufficient (and taking more that might possible be relevant would probably imply taking an efficiency and persistence hit)

  • Reduction pipeline currently not doing great job at sky subtraction and telluric correction

  • Relatively little effort dedicated to date

  • Not yet clear if calibration data is sufficient

  • Have been using 35/35 sky/telluric as opposed to original plan of 25/25

32

Dec 8, 2011


Status of apogee data products reduction and analysis

Data inspection / QA Michael Hayden, Jon Holtzman (NMSU)

  • Data is now being routinely processed (although almost certainly not optimally!)

  • quick manual construction of pre-processing file (check on plate/exposure assocation, cal frames, etc.) done morning after observing (~15 minutes) based on observing logs

  • Raw data cubes downloaded from SAS, generally by night following observing

  • Reduction started second day after observing, processed through 3D->2D, 2D->1D, sky correction, dither combination, RV measurement in 6-8 hours per plate, all plates reduced simultaneously on multiple processors

  • Daily calibration data processed in parallel (not yet quite routinely implemented)

33

Dec 8, 2011


Status of apogee data products reduction and analysis

Data inspection / QA (2) Michael Hayden, Jon Holtzman (NMSU)

  • Data reduction pipeline produces web pages for quick inspection / QA

  • Summary MJD QA page:

    • Highlights any frames for which no reduced frame exists

    • Frames with bad checksums

    • Missing sequence numbers

    • plot of zeropoints for all exposures, continuum sky levels for all exposures

    • Table of all science exposures, with derived S/N, zeropoints, attempts at identifying missing and faint fibers

  • Summary web page for each plate:

    • Zeropoint, continuum sky, and S/N plots vs mag for every exposure

    • Maps of zeropoints, sky as f (zeta, eta)

  • Web page of all dither-combined spectra and error spectrum for each plate

    • Links to RV page showing best matching spectrum

  • Some of these are getting looked at now

  • Need to develop specific checklist of what should be looked at (may motivate some reorganization of information on pages), but I think we are very close

  • This checks mostly issues with conditions, data issues, and data reduction issues, thus indirectly instrument health

34

Dec 8, 2011


Status of apogee data products reduction and analysis

Data inspection / QA (3) Michael Hayden, Jon Holtzman (NMSU)

  • Instrument health montoring through daily calibration monitor output: not yet fully implemented, but many pieces in place (few days work)

  • Wavelength calibration: start/end wavelengths, wavelength solution vs. fiducial solution

  • LSF: line width monitoring

  • Dither pair separation (should be included in plate pages)

  • Daily flat vsfiducial flat

  • Daily dark vsfiducial dark

  • Cartridge flats vsfiducial (perhaps should be included in plate pages)

  • Need to develop specific checklist of things to look at and simple web interface (high priority, getting close to implementation)

  • Increment of survey reduced data statistics

  • RV histogram for plate

  • Sky map of observed plates

  • S/N histogram

  • Delta mag histogram

35

Dec 8, 2011


Status of apogee data products reduction and analysis

Possible areas for improvement Michael Hayden, Jon Holtzman (NMSU)

  • EFFICIENCY

  • Shortening default exposure time

  • Implement variable exposure time based, e.g., on quicklook

  • Use of twilight time

  • Additional cartridges or nighttime replugging of a cartridge

  • Reduction of number of sky and/or telluric fibers

  • Reducing/eliminating daily sky exposures

  • Developing refined S/N criteria, e.g., based on actual ASPCAP results

  • ANALYSIS

  • Develop methods for addressing persistence

  • Improve extraction methods

  • Improve telluric correction, and perhaps, sky subtraction

  • Improve construction of dither-combined frames

  • Implement combination of multiple visit spectra

36

Dec 8, 2011