NWO Meeting 24 May 04
a Virtual Survey System
AstroWise- OmegaCAM – Lofar – Euro-VO a Virtual Survey System
AstroWise- OmegaCAM – Lofar – Euro-VO
Edwin A. Valentijn
Edwin A. Valentijn
statement statement
• Big steps with small groups of people
– serve individual research groups – don’t set out to heal the world – sociology
• Astronomers and very large db
– 1989 ESO-LV 30.000 images – Work station
– Sloan Digital Sky Survey– MicroSoft Research – AstroWISE – Euro VO
• Big steps with small groups of people
– serve individual research groups – don’t set out to heal the world – sociology
• Astronomers and very large db
– 1989 ESO-LV 30.000 images – Work station
– Sloan Digital Sky Survey– MicroSoft Research
– AstroWISE – Euro VO
NWO Meeting 24 May 04
VST VST
OmegaCAM Detectors OmegaCAM Detectors
• Science array 1 x 1 degree, 32 CCDs
– 15 m pixels – 0.21 arcsec/pixel
– Marconi (former EEV) 2k x 4k= 8 Mpix – 16k x 16k pixels = 256 Mpix
• Auxiliary CCD’s – 4 CCDs
– For guiding
– Image analysis
• Science array 1 x 1 degree, 32 CCDs
– 15 m pixels – 0.21 arcsec/pixel
– Marconi (former EEV) 2k x 4k= 8 Mpix – 16k x 16k pixels = 256 Mpix
• Auxiliary CCD’s – 4 CCDs
– For guiding
– Image analysis
NWO Meeting 24 May 04
Paranal Paranal
Mid 2005 Mid 2005
+ 3 year + 3 year
Lofar Lofar
IBM- Blue Gene/L
IBM- Blue Gene/L
NWO Meeting 24 May 04
Large Data Volume Large Data Volume
• Wide-field imaging = vast amounts of data
– VST maps equivalent of Southern sky in 3 years
30 min exp, 300 nights/y.
100 Tbyte image data Tbytes source list data
• Wide-field imaging = vast amounts of data
– VST maps equivalent of Southern sky in 3 years
30 min exp, 300 nights/y.
100 Tbyte image data Tbytes source list data
• Handling of the data is non-trivial
– Pipeline data reduction – Calibration
– Image comparisons/combinations – source lists / catalogues
– Visualization
• Science can only be archive based
• Handling of the data is non-trivial
– Pipeline data reduction – Calibration
– Image comparisons/combinations – source lists / catalogues
– Visualization
• Science can only be archive based
ASTRO-WISE Mission Virtual Survey System
ASTRO-WISE Mission Virtual Survey System
• Environment that provides systematic and controlled
– Access to all raw and calibration data
– Execution and modification image/calibration pipelines – Execution of source extraction algorithms- catalogues – Archiving or regenerate dynamically
– federated to link different data centers
• Environment that provides systematic and controlled
– Access to all raw and calibration data
– Execution and modification image/calibration pipelines – Execution of source extraction algorithms- catalogues – Archiving or regenerate dynamically
– federated to link different data centers
• Dynamical archive continuously grows
-- small or large science projects
– generating and checking calibration data – sharing methods, scripts
• Dynamical archive continuously grows
-- small or large science projects
– generating and checking calibration data – sharing methods, scripts
raw pixel data pipelines/cal files catalogues
raw pixel data pipelines/cal files catalogues
NWO Meeting 24 May 04
New technologies-paradigm New technologies-paradigm
• raw data centralized – on the fly reprocessing
• sociology- project management
• ++
• Python
• parallel – IP - seti@home
• user tunable pipelines
• ++ db
• 5 LS – 5 lines script
• raw data centralized – on the fly reprocessing
• sociology- project management
• ++
• Python
• parallel – IP - seti@home
• user tunable pipelines
• ++ db
• 5 LS – 5 lines script
development issues development issues
• Indexing Terabytes on the sky
• Associating entries
• World wide linking entries (referencing) - registries
• Filtering
• Visualisation - imaging
• Visualisation -multidimensional data
• VO world
• Indexing Terabytes on the sky
• Associating entries
• World wide linking entries (referencing) - registries
• Filtering
• Visualisation - imaging
• Visualisation -multidimensional data
• VO world
NWO Meeting 24 May 04
VST - Virtual Survey Telescope
VST - Virtual Survey Telescope
Source List
RA0, Dec0 t0
V filter
Source List V filter
t2 RA0?, Dec0?
Source List V filter
RA0?, Dec0?
t1
S
S
C I
T S E
A
A
O
B filter
RA0, Dec0
Source List
Source List V filter
RA0, Dec0
S
S
C I
T S E
A
A
O
S
S
C I
T S E
A
A
O Source
List B filter
RA0, Dec0
Source List B filter
RA1, Dec1
Source List V filter
RA0, Dec0
Source List B filter
RA1, Dec1
Source List B filter
RA0, Dec0
Source List B - V
RA0, Dec0
Source List V filter
RA0, Dec0 t0
Source List V filter
RA0?, Dec0?
t1
Source List V filter
t2 RA0?, Dec0?
Source List V filter
RA0?, Dec0?
tn
Proper motion Red quasars
Supernovae
