millennium data dissemination n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Millennium Data Dissemination PowerPoint Presentation
Download Presentation
Millennium Data Dissemination

Loading in 2 Seconds...

play fullscreen
1 / 21

Millennium Data Dissemination - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Millennium Data Dissemination. possible extensions how ambitious can/should we be?. FOF groups and Subhalos. Mock images. Raw data: Particles. Density fields. Subhalo merger trees. Synthetic galaxies (SAM). Mock catalogues. MRObs. Scaling . Simulation Particles . FOF Groups.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Millennium Data Dissemination' - dakota


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
millennium data dissemination

Millennium Data Dissemination

possible extensions

how ambitious can/should we be?

MPA institute seminar

slide2

FOF groups and Subhalos

Mock images

Raw data:

Particles

Density fields

Subhalo merger trees

Synthetic galaxies (SAM)

Mock catalogues

Millennium Workshop

mrobs
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

FOF Group

Finder

SUBFIND

Tree Builder

Gadget

SAM

SSP

SExtractor

SkyMaker’

MoMaF’

SAM Galaxies

SExtractor Catalogues

Synthetic Images

Light-cone Galaxies

IGM

PSF, Noise,

Background

Millennium Workshop

raw data
Raw data
  • Serve particles from Millennium simulations
    • 10 billion/snapshot, ~25TB
  • from database?
    • with spatial queries
    • links to halos/FOF groups
  • through JobRunner services
    • cutout
    • gridding + FFTs etc

MPA institute seminar

other simulations
Other simulations?
  • Hydro Example:
    • http://gavo.mpa-garching.mpg.de/HydroClusters
    • online visualisation: http://galformod.mpa-garching.mpg.de/phoxybrowser
  • Others
    • Millennium+gas?
    • Eagle
  • How?
    • In DB ok for objects
    • Visualisation support for “fluffy things”

MPA institute seminar

mrobs1
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

FOF Group

Finder

SUBFIND

Tree Builder

Gadget

SAM

SSP

SExtractor

SkyMaker’

MoMaF’

SAM Galaxies

SExtractor Catalogues

Synthetic Images

Light-cone Galaxies

IGM

PSF, Noise,

Background

Millennium Workshop

group catalogs
Group catalogs
  • Have
    • FOF
    • SUBFIND subhalos
    • Mhalo merger trees
    • Dhalo merger trees
  • More properties
    • shapes, density profiles (ala AIP, or ala MR-XXL): good for lensing
    • others?
  • More?
    • BDM, NEXUS, ORIGAMI, …
    • All of Knebeetal
  • LSS objects?
    • Voids, filaments, walls in the DB
    • “more precise” environmental queries possible
    • per grid cell or as object (“stacking voids”)

MPA institute seminar

mrobs2
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

FOF Group

Finder

SUBFIND

Tree Builder

Gadget

SAM

SSP

SExtractor

SkyMaker’

MoMaF’

SAM Galaxies

SExtractor Catalogues

Synthetic Images

Light-cone Galaxies

IGM

PSF, Noise,

Background

Millennium Workshop

sam catalogues
SAM catalogues
  • More models
    • Durham, MPA
  • Complete MR-XXL?
    • ~200x Millennium, 80TB
  • Extensions

MPA institute seminar

sfh s
SFH-s
  • Allows SEDs from post-processing
  • how to store?
    • rows vs arrays (in SQLServer)
    • functions (easy in SQLServer):
  • create spectra
    • “exact” observer frame bands
  • PCA amplitudes in the database
    • find nearby galaxies for given observed spectrum

MPA institute seminar

sam online
SAM online
  • GALFORM online !!!
    • http://galformod.mpa-garching.mpg.de/dev/jobrunner/
    • on millimil
  • Do we want others?
    • Galacticus?
  • Need constraints, validated models.
    • How afraid should we be of “bad science”?
  • SAM online on Millennium?
    • input to MRObs pipeline

MPA institute seminar

mrobs3
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

FOF Group

Finder

SUBFIND

Tree Builder

Gadget

SAM

SSP

SkyMaker’

SExtractor

MoMaF’

SAM Galaxies

SExtractor Catalogues

Synthetic Images

Light-cone Galaxies

IGM

PSF, Noise,

Background

Millennium Workshop

light cones
Light cones
  • In Database: not just from MPA!
    • common model
    • linked to SAMs
  • More
    • deep: new code for Millennium-II
    • all sky: MR-XXL
    • with scaling!
  • What properties needed?
    • who wants them?
    • what for (lensing?)?
  • Light cone building online
    • Will soon make our MoMaF available on JobRunner
    • new code in preparation (for MR-II etc)

MPA institute seminar

mrobs4
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

FOF Group

Finder

SUBFIND

Tree Builder

Gadget

SAM

SSP

SExtractor

MoMaF’

SkyMaker’

Light-cone Galaxies

SAM Galaxies

SExtractor Catalogues

Synthetic Images

IGM

PSF, Noise,

Background

Millennium Workshop

mrobs5
MRObs
  • Store of images will keep growing
    • Overzier!
  • Create images on demand
    • telescope models custom?
  • Post processing as well?
    • Sextractor
    • other?
  • Improvement galaxy images
    • realistic structure (weak lensing systematics, strong lensing structure)
    • from simulations ?
    • form observations?

MPA institute seminar

mrobs6
MRObs

Scaling

Simulation Particles

FOF Groups

(Sub-)Halos

Halo Merger Trees

SUBFIND

FOF Group

Finder

Tree Builder

Gadget

SAM

SSP

Lensing

SExtractor

MoMaF’

SkyMaker’

Light-cone Galaxies

SAM Galaxies

SExtractor Catalogues

Synthetic Images

IGM

PSF, Noise,

Background

Millennium Workshop

lensing
Lensing
  • Under construction
    • with Ben Metcalf
  • Online?
    • Why not, JobRunner will handle it.
  • How much?
    • what is required for weak lensing?

MPA institute seminar

even more
Even more
  • L-Galaxies online++
    • updating physics modules?
    • MCMC mode?
  • Dark matter light cones?
    • for lensing
    • what do we need? particles, fields?
  • Interface
    • dashboard (M.Egger)
    • improve SQL interface (Roys talk)

MPA institute seminar

galformod infrastructure
Galformod infrastructure
  • Currently
    • 12+1 12-core nodes
    • 180 TB raw storage
    • 20TB DB storage
  • What do others have?
    • AIP ~500TB
    • JHU ~5PB (?)
  • What do we want?

100 billion …

MPA institute seminar

what do we want need
What do we wantneed?
  • Depends on answers to all previous questions
  • What can we afford?
    • Where can we get the money?
  • Where do we put “it”?
    • RZG?
  • Pool our resources together?
    • beyond mirror
    • distributed over Virgo nodes?
  • Larger scale collaborations (EUDat)?
    • see Hermann’s talk

MPA institute seminar