Recap of data access methods see chapter 1 of data analysis guide and youtube videos
This presentation is the property of its rightful owner.
Sponsored Links
1 / 7

Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos! PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on
  • Presentation posted in: General

Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos!. David R. Ardila User Support coordinator / NHSC Archive Scientist. Data Management. D ata management in Herschel i s complicated by the size of datasets

Download Presentation

Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos!

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

Recap of Data Access MethodsSee Chapter 1 of Data Analysis Guide andyoutube videos!

David R. Ardila

User Support coordinator / NHSC Archive Scientist


Data management

Data Management

  • Data management in Herschel is complicated by the size of datasets

  • Everything is about making data management efficient

  • Do you really need big data sets in your disk?


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

  • For most people, this is all you need.

  • Gets references to the data in HIPE and downloads data as needed

  • Useful for

    • Quick inspection

    • Data reduction

  • Does not save, unless you do it explicitly*

  • *of course, there is a way to override this!

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

HIPE


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

Tar file in

local disk

getObservation()

Use Navigator

untar

Directory in

local disk

  • HSA to local disk

  • May be slow

  • But then you have the data

  • It allows you to bypass HIPE


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

getObservation(…,useHsa=True)

Product Browser

HSA

getObservation()

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation()

Use Navigator

untar

  • Pools are good if:

  • You have a lot of data

  • That needs reprocessing

Directory in

local disk


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

HSA

getObservation(…,useHsa=True)

Product Browser

getObservation()

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation(…, useHsa=True, save=True)

getObservation()

Use Navigator

untar

getObservation()

Product Browser

myHSA

A type of pool.

Unmodified data (or links) from HSA

Directory in

local disk


Recap of data access methods see chapter 1 of data analysis guide and youtube videos

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

getObservation()

Product Browser

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation(…, useHsa=True, save=True)

getObservation()

Use Navigator

untar

getObservation()

Product Browser

myHSA

A type of pool.

Unmodified data (or links) from HSA

Directory in

local disk


  • Login