recap of data access methods see chapter 1 of data analysis guide and youtube videos n.
Download
Skip this Video
Download Presentation
Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos!

Loading in 2 Seconds...

play fullscreen
1 / 7

Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos! - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos!. David R. Ardila User Support coordinator / NHSC Archive Scientist. Data Management. D ata management in Herschel i s complicated by the size of datasets

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Recap of Data Access Methods See Chapter 1 of Data Analysis Guide and youtube videos!' - maxine


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
recap of data access methods see chapter 1 of data analysis guide and youtube videos

Recap of Data Access MethodsSee Chapter 1 of Data Analysis Guide andyoutube videos!

David R. Ardila

User Support coordinator / NHSC Archive Scientist

data management
Data Management
  • Data management in Herschel is complicated by the size of datasets
  • Everything is about making data management efficient
  • Do you really need big data sets in your disk?
slide3

For most people, this is all you need.

  • Gets references to the data in HIPE and downloads data as needed
  • Useful for
    • Quick inspection
    • Data reduction
  • Does not save, unless you do it explicitly*
  • *of course, there is a way to override this!

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

HIPE

slide4

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

Tar file in

local disk

getObservation()

Use Navigator

untar

Directory in

local disk

  • HSA to local disk
  • May be slow
  • But then you have the data
  • It allows you to bypass HIPE
slide5

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

getObservation(…,useHsa=True)

Product Browser

HSA

getObservation()

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation()

Use Navigator

untar

  • Pools are good if:
  • You have a lot of data
  • That needs reprocessing

Directory in

local disk

slide6

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

HSA

getObservation(…,useHsa=True)

Product Browser

getObservation()

Product Browser

“Send to External Application”

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation(…, useHsa=True, save=True)

getObservation()

Use Navigator

untar

getObservation()

Product Browser

myHSA

A type of pool.

Unmodified data (or links) from HSA

Directory in

local disk

slide7

Local Pools

(working directories

in the local disk)

saveObservation()

saveProduct()

Save Products GUI

HSA

getObservation(…,useHsa=True)

Product Browser

“Send to External Application”

getObservation()

Product Browser

“Retrieve Products”

HIPE

HIPE

Tar file in

local disk

getObservation(…, useHsa=True, save=True)

getObservation()

Use Navigator

untar

getObservation()

Product Browser

myHSA

A type of pool.

Unmodified data (or links) from HSA

Directory in

local disk