Issues related to designing a VO for Heliophysics

1. Issues related to designing a VO for Heliophysics Bob Bentley (UCL-MSSL) 13 June 2007 VOiG Conference, Denver CO, USA

2. Outline • Examine some of the items needed to facilitate a virtual observatory for heliophysics • Identify deficiencies and suggest solutions • Talk evolved as I was preparing it… • Started looking at proton events and GLEs because of space weather effects related to aviation • Found problems with the metadata

3. Heliophysics • Heliophysics explores the Sun-Solar System Connection • An extension of the study of space weather (SWx++) • Heliophysics sits in the boundary between two communities • Astrophysics • An understanding of solar phenomena helps in the understanding of stellar observations • Planetary sciences (including Geosciences) • Solar activity can influence the environment on/around the planets • The discipline must be aware of the need to support the interests of both communities • A virtual observatory that supports Heliophysics must facilitate access to data from a number of communities • Solar, heliospheric, magnetospheric and ionospheric physics • As such, it is in essence a next generation VxO

4. Heliophysics • The communities that constitute heliophysics have evolved independently over decades and even centuries • Little or no coordination of how each has evolved • Each has very different ways of describing, storing and exploiting the data from their observations • The desire to solve science problems that span disciplinary boundaries is driving the need to provide combined access to these data • To do this, we need to find ways to: • Tie the data together through searches across all the datasets • Present any results in a form that does not require a detailed understanding of each discipline • This requires the re-evaluation of the capabilities provided within each community and some correctative action

5. Types of metadata • To facilitate a VxO for heliophysics we need to examine the metadata that is required. • There are many ways these can be described, one is: • Search metadata • Metadata used to identify time intervals and sets of data of interest • Observational metadata • Metadata used to describes the observations, e.g. FITS headers • Storage metadata • Metadata that describes how the data are stored and accessed • Administrative metadata • Metadata that allows the system to exploit the available resources • The rest of the talk will concentrate on issues related to the first two sets

6. Searches • In heliophysics, we are interested in how an event on or near the solar surface can propagate through the heliosphere and affect a planetary environment • Searches should identify interesting time intervals based on a combination of event, features, etc. metadata • Each community of some from of these data • There are concerns about the quality and integrity of the metadata and whether it is adequate to support the searches we would like to undertake

7. Solar search metadata • Phenomena occur on or near the solar surface • Event data gives time and location of phenomena • Feature data provides details of location and size of structures that may be relevant • Time information can be expressed in many ways • Essentially these are the same, with simple transformations • Spatial information can be expressed in terms of: • Coordinates in the observing frame – e.g. arcsecs from disc centre • Coordinates on the rotating body of the Sun – Carrington coords. • The position of the observer was ignored for the most part • Helio-seismology is an exception • In the bigger picture of heliophysics, also need to include the viewing perspective

8. Other search data • Observation of phenomena in the heliosphere and near/on planets are more complex • For in-situ observations in the heliosphere • Time is when a phenomena affected (passed) the observer • Position of the observer relative to the Sun is key to understanding • When the in-situ observations are made on/near a planet • Position of the observer relative to the planet is also important • Relating events that are defined from in-situ data to those on/near the Sun requires an understanding of how events propagate • Details of the velocity structure of CMEs and the solar wind are not easy to determine…

9. Simple, but not so simple • In principle this all seems fairly obvious, but lets look in detail at some common solar event data • On 20 January 2005 there was an X7.1 flare that was intensely geo-effective. • The flare was associated with particle event and a CME; it was also observed by ground-level neutron monitors – a GLE. • Many superlatives were used to describe the event • "The solar energetic particle event of January 20 2005 has been called, by some measures, the most intense in 15 years..." (Mewaldt et al., 2005) • ”The fastest rising SEP event of current cycle [cycle 23]" (Rawat et al., 2006) • ”The most spectacular [solar event] of the Space Age" (Tylka et al., 2006) • ”The largest GLE [GLE 69] in half a century" (Bartol Research Institute) • But event is absent from the NOAA SEC list of "Solar Proton Events Affecting the Earth Environment" • When you look at the data and how lists are created, you realize that the lists are deficient in several ways • Humans and SmFCACs can understand what happened, but • it is harder for machines...

10. X7.1 of 20 Jan 2005 • The event was one of several from AR 10720 • Two other X class flares and several M class flares occurred in previous 3 days; others before this

11. X7.1 of 20 Jan 2005

12. X7.1 of 20 Jan 2005 • At the time of the event, the proton levels had not returned to normal after previous events • The criteria fails to recognize a new event • NOAA lists event on 16 Jan • The X-ray data also suffers from problems • The end of an event is defined by when the counts drop to 50% • New events can “interrupt” existing events • The shape and true duration of the decay phase are lost • NOAA gives start 0636; end 0726 • Not all locations are tagged!! • Significant brightenings seen on images not declared as flares

13. Some of the problems • That major events can be “missed” is worrying and makes automated searches difficult • A search for long duration events would yield spurious results • Since the locations of all flares are not known, it is impossible to know if they will be geo-effective • Instrument flare lists have gaps – nights, off times, etc. – but the reason for a null result is not included

14. Improving event data • Existing flare lists can give a distorted picture what has occurred • Such deficiencies make it difficult for non-experts to use them • The community “knowledge” is not written down • Need to re-evaluate and regenerate the event data in all domains with the idea that the data will be used in a joint search across the domains • Ensure events more accurately described • Include information that might explain null results

15. Observation metadata • Metadata related to observations gives information about how the observation was made, etc. • There are often quality issues related to the metadata that is provided – parameters sometimes missing, or wrong • In solar data, space-based observations much better described than their ground-based counterparts • Researchers often used to deficiencies in the data in their own domain • Difficult for machines to handle if it is not quantified properly • VxOs can sometimes develop ways of patching the gaps • What do we do with this information? How is it shared?

16. Observation metadata • In solar physics we have tried to introduce standards • SOHO made good start with their keyword document • EGSO enhanced concepts with its data model • Situation better than it was but adoption still not universal • Even some problems within SOHO…

17. Standards • Need to extend the use of standards to all domains so that all future data are more compatible with the VxOs • The situation has changed with enhancements in technology; providers need to ensure they are more compliant • VxO will need to handle problems with the older data; providers cannot be expected to do it • Standards need to be developed in collaboration with the community and funding agencies • Core part that is required • Additional information that may be specific to a domain • Other information that the instrument team wants to add • In developing standards need to draw on the experience gained within the general VO community and adopt the best ideas available

18. Conclusions • Developing a virtual observatory to support heliophysics will not be simple • Some of the possible problems have been highlighted • To address them, we need to engage the communities in all the domains that constitute heliophysics and develop standards that will facilitate the process