Gijs Verdoes Kleijn OmegaCEN, Kapteyn Astronomical Institute, University of Groningen

1. ASTRO-WISE Science“A new approach to astronomical archiving & researching for the data-flooded era” Gijs Verdoes Kleijn OmegaCEN, Kapteyn Astronomical Institute, University of Groningen

2. The OmegaCEN team at Kapteyn • Edwin Valentijn (lead) • Kor Begeman • Danny Boxhoorn • Erik Deul (Leiden) • Ewout Helmich • Philippe Heraudeau • John MacFarland • Michiel Tempelaar • Gijs Verdoes Kleijn • Ronald Vermeij • Willem Jan Vriend (Lofar) • Kovac, Schneider, Sikkema (PhDs)

3. Archiving your vision • Camera in front of eye: • resolution ~5/60 degree (my eye) • Field of view 902 degree • 2x2 pixel sampling per resolution element • #pixels= 21602 • Dynamic range: ~103-5: take 216 • One image ~9Mbyte • One image/sec for 70years: 18Pbyte • Storage cost (at 0.33euro/Gbyte)~ 6million euro • ~1010-11 neurons Brain does something smart… ……intensive linking is a key ingredient.

4. If vision archive were implanted for re-analysis • Applying improved analysis • (Raw data: improved calibrations as well) • Analysis of previously deemed uninteresting parts of data • Variability analysis • if archives from different persons are combined • Denser coverage in space+time • 3D construction of 2D view • If fly’s UV eye archives are added: • Same object at different wavelengths Aim: exploit data for purposes or in ways not yet conceived ... ... flexible, internally linked, information system required

5. Key properties for intelligent information system • Adaption: facilitate changes • Changes due to improved encoded methods • True physical changes of parameter values (e.g., change in instrument/atmospheric properties) • Improved insight in 2 or 3 • Learning: “more and better” • take advantage of existing results (from you or others) • Fast (re-)reduction and (re-)analysis • From quick-look to high-quality results • Not only more but also better data over time: ‘accumulation of knowledge • Anybody • Small Individual research projects • Large projects with many collaborators • Everywhere: federation • federation of storage and computing capabilities • Scalability • No limits due to design for storage, databse power/storage processing power,… A federated ‘brain’ interacting with many users

6. The Astro-Wise Environment a new archive/research tool for astronomical wide-field imaging • Status • environment is working • First paper using Astro-Wise based results is out • Expansions and improvements on-going

7. Data server: images calibration & science From raw to reduced CPUs +algorithms The Astro-Wise Environment User Python prompt (awe>) Web interfaces “Interpreting” • Database • metadata of images • derived data from images (source lists) • =‘spider spinning AWE web’ • Contains ALL input/output “Archiving” “Processing”

8. Key properties for intelligent information system • Adaption: facilitate changes • Changes due to improved encoded methods • True physical changes of parameter values (e.g., change in instrument/atmospheric properties) • Improved insight in 2 or 3 • Learning • take advantage of existing results (from you or others) • Outdated not-yet existing data • Fast (re-)reduction and (re-)analysis • Not only more but also better data over time: ‘accumulation of knowledge‘ • Anybody: • Each piece of information carries tag of ‘ownership’ • Small Individual research projects • Large projects with many collaborators • Anywhere: federation: • Own developed compute-grid and storage-grid • Nodes:active={Kapteyn, Bonn} almost={Munich, Paris,Naples}, in progress={Nijmegen, Leiden, Santiago}) • Scalability: performance proportional to I/O speed and processing power. A federated ‘brain’ interacting with many users

9. Paradigm shift

10. Cen A Science projects with AWE 2dF • PhDs Sikkema, Kovac, Schneider: galaxy surveys with WFI, WFC, MDM • Test projects • Variable sources around CenA • light curves (Δmag~0.03) for 2x104 objects around Centaurus A • Valentijn • Asteroid detection • Detections in WFI image from catalog of ~105 numbered asteroids • Jeffrey Bout (student), myself Asteroids

11. Large public data projects with AWE using OmegaCAM at VST • KIDS (PI: Kuijken): ESO Public Survey • Weak lensing; high-z QSOs; galaxy/cluster evolution; baryon oscillations; • 5000 deg2 u,g,r,i, ~500 nights KIDS South KIDS North

12. GTO science with AWE using OmegaCAM at VST • OmegaWhite (PI: Groot): • discover Galactic Population of ultracompact binaries from periodic (<2hour) light curves • 150 deg2 @ b=±5o; Sloan g’ 40sec exposures & additional ugriz’ coverage.

13. GTO science with AWEusing OmegaCAM at VST • OmegaTranS (Saglia; Snellen; Alcala) • Searching for planet transits (15-20 new transits expected in first year) • ~1 order more powerful than OGLE-III

14. GTO science with AWEusing OmegaCAM at VST • VESUVIO (PIs: Valentijn, Capaccioli) • Galaxy/cluster evolution • Horologium Supercluster: 100 deg2 medium deep (r'<25mag); ugriz • Hercules Supercluster: 12 deg2 deep ugriz+Hα

15. Key ingredients to achieve the Astro-Wise environment • Strict global data acquisition and processing model • data model -> object model • storing all I/O in single (distributed) database • Database environment exploits • OOP inheritance (Python) • Complete linking (associations, references)

16. Conclusions • New analysis environment for wide-field imaging in operation by OmegaCEN • Key ingredient: dynamic database • Containing all I/O • Fully linked data • Versatile: could be used for other kinds of data • Collaboration with LOFAR on-going • Large science projects with AWE when OmegaCAM starts operations (early 2007) • To find out more • Visit : www.astro-wise.org; • download/get now 2 page overview article

17. Contribution to LOFAR project