slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley PowerPoint Presentation
Download Presentation
Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley

Loading in 2 Seconds...

play fullscreen
1 / 21

Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley - PowerPoint PPT Presentation


  • 379 Views
  • Uploaded on

PySALT: SALT’s PyRAF Pipeline. Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley. Outline. Introduction to PySALT: the SALT data reduction and analysis software package PySALT and the SALT Pipeline Reducing SALT data

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley' - EllenMixel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

PySALT: SALT’s

PyRAF Pipeline

Steve Crawford

crawford@saao.ac.za

Collaborators: Martin Still, the SALT astronomers, and David Buckley

outline
Outline
  • Introduction to PySALT: the SALT data reduction and analysis software package
  • PySALT and the SALT Pipeline
    • Reducing SALT data
    • Slotmode analysis tools
  • The Future of PySALT development

SALT Goettingen Board Meeting

pysalt optimizing performance
PySALT: Optimizing Performance

Purpose of the SALT Science Software Development:

Science quality data reductions of SALT observations

Data analysis tools for the unique modes of SALT

Algorythms to handle the data archiving and timely distribution to principle investigators

Advantages:

Maximize scientific returns for the telescope.

Minimize redundant work

3. Establish a framework for archiving and distribution

4. Provide for a method for fast response science

How to interact with the virtual observatory

Provide software for unique modes of SALT

SALT Goettingen Board Meeting

maximizing scientific return
Maximizing Scientific Return

Steps to Publishing a Paper:

Observe an Object

???

Publish!!!

Minimizing the steps in ??? will greatly increase the scientific productivity of SALT

Minimize time spent

reducing data

(we’re all doing the same things)

Spend more time on data

analysis

=

Different

Publications

More

Publications

Plus having standard reductions open up new observations to

Scientist that may never have considered attempting.

SALT Goettingen Board Meeting

what are the needs of the salt pipeline
What are the needs of the SALT pipeline?
  • Flexibility to handle a large range of data including unique modes such as high speed imaging and Fabry-Perot Spectroscopy (not to mention high-speed Fabry-Perot polarimatry)
  • Ability to process that data in real time
  • To deliver high quality science data to the PI’s
  • An automated system included the ability to analyze data quality and reduction status

SALT Goettingen Board Meeting

development langauge python pyraf
Development Langauge: Python/PyRAF
  • PyRAF provides a similar environment and access to IRAF
  • Python is a well developed, well-supported programming language with run-time compilation and scripting ability.
  • Free, open source software
  • Large and growing library of numerical and image processing software.
  • Ability to wrap and interact with a large number of other code bases

SALT Goettingen Board Meeting

pysalt
PySALT

PySALT is the main software package for SALT data

reduction and analysis. Along with library containing tasks

for error handling, I/O, interaction with the database, and

other shared tasks, the PySALT package currently contains

three main software packages:

First Public Software Release:

28 April 2008

PIPETOOLS

Tasks to automate

the data reduction

And data handling

SALTRED

Basic CCD data

Reductions

SLOTTOOLS

Slotmode photometry

And analysis tools

SALT Goettingen Board Meeting

pipetools
PIPETOOLS

Prepare Data

The Pipetools package is designed

to handle the daily reduction, archiving, and distribution of the data. Each day at 10:30 am, the pipeline automatically begins processing the data from the previous night. An SA will verify the data quality before beginning a separate task that will archive the data, place the data on an ftp server, and notifying the PI that their data is ready for retrieval.

Clean Images

Create documentation

Archive

FTP

Email

SALT Goettingen Board Meeting

saltred
SALTRED

Steps in SALT CLEAN

Prepare Data

General preparation of the data

Before running other corrections.

Correct for the different gain levels

From each of the amplifiers

According to the database file

Gain Correction

SALTslot.py

Apply the cross-talk correction

Xtalk correction

Overscan and trim the data. Can

Also apply the master bias but

Currently does not.

Bias correction

CR Clean

Cosmic Ray clean the data

Mosaic the multi-extension data

Into one file.

Mosaic images

SALT Goettingen Board Meeting

performance
Performance
  • Fast reduction of SALT slot mode:
    • Cleans the data at a rate of >50, 000 images per hour
    • Reduce data as fast as the data are taken
  • Due to the purchase of a dedicated pipeline computer with optimized I/O performance, a typical night’s worth of data can be fully reduced in under 20 minutes (~2.5 GB) with a heavy night’s worth of data in 40 minutes (6 GB)
  • The Bottleneck: Internet Bandwidth!

SALT Goettingen Board Meeting

slottools
SLOTTOOLS

SLOTPHOT

Main task that measures

photometry for slotmode

observations. The taks will:

Subtract the global sky background

Follow the drift of one of the two sources

Measure the photometry for two sources via four different methods

Slotview

Task that provides an

interactive environment for

viewing the lightcurve and

data produced by Slotphot.

In addition to repeating the

photometry the user can also

save the data, change the

targets, delete sources, and

assess the quality of the light

curve.

SALT Goettingen Board Meeting

documentation
Documentation

Documentation will be supplied in five parts:

  • IRAF-like help files for each task
  • Development documents for each task
  • Reduction recipes for each mode
  • Planning documents for each package.
  • Manuals for each package. Slottools manual distributed with the most recent software release.

SALT Goettingen Board Meeting

the future of pysalt
The Future of PySALT

Basic Reductions

Task for doing the basic reductions which are common for most modes. Still need tools for flat fielding and fringe correction.

Extended Reductions:

Tasks for reducing each

of the individual modes.

It will be important to produce reduced data for long-slit, multi-slit, slot mode, drift-scanning, and many more

Analysis Tools: It will be important to develop tools for analyzing the wide range of modes from SALT data. For example: saltslot.py

SALT Goettingen Board Meeting

development goals for pysalt
Development Goals for PySALT

A primary goal of the pipeline is to supply science-quality reduced data to the principle investigator. At the current time, our planned development include the following modes:

Reduced images with astrometric solutions ready for relative photometry

Fullframe

Slotmode

Fullframe

Slotmode

Reduced images ready for relative photometry

Wavelength calibrated spectra

ready for further analysis

Long slit

Long slit

MOS

MOS

Wavelength and polarized calibrated spectra

ready for further analysis

Polarimetry

Slot Spectra

Slot Spectra

Polarimetry

Wavelength calibrated data cube

ready for further analysis

Fabry-Perot

Fabry-Perot

SALT Goettingen Board Meeting

salt community
SALT Community

The SALT Science webpages are currently a static format containing the data reduction and analysis software

Interactive community that allows for a clearinghouse for SALT Science

Method for accessing

Data through the web

manager

Interaction with the

VO and online

Analysis tools

Fourm for discussing

SALT Science

SALT Goettingen Board Meeting

ask not what the salt pipeline can do for you
Ask not what the SALT pipeline can do for you…
  • Feedback--if you get data, give us feedback. Is the data good enough? Is there anything you would change? Is there something you need or would be great to add?
  • Adopt python/PyRAF as software environment
  • Share algorithms, software,
  • Contribute support

Triton and Neptune from SALT

20080517--N. Loaring

SALT Goettingen Board Meeting

summary
Summary
  • The SALT PyRAF pipeline has been processing data successfully since Oct ‘07
  • Software currently include basic CCD reductions plus analysis tools for slotmode and was released April 28.
  • Future plans include reductions and analysis tools to support most of the major modes of SALT
  • Next major released planned for Dec ‘08 and will include tasks for long slit reductions.

SALT Goettingen Board Meeting

wet campaign d buckley salt
WET CampaignD. Buckley (SALT)

SALT Goettingen Board Meeting

resources
Resources
  • SAAO Pipeline Development:
  • http://www.salt.ac.za/science-support/pyrafiraf-package/developers-page/
  • PyRAF
  • http://www.stsci.edu/resources/software_hardware/pyraf
  • Python
  • http://www.python.org/doc/
  • http://www.diveintopython.org/

SALT Goettingen Board Meeting