Metadata set definition, automated data collection, storage, quality assurance checks and uncertainty estimates, information building and model data integration.
The use of a new common central information system to combine in-situ and spatial data/metadata and to enhance the harmonisation of exchange mechanisms will be investigated and tested within the Action network. Essentially, the system must support the data life cycle of spectroradiometric data, namely: data acquisition, data processing, metadata augmentation, information building, and do so in a ‘smart’ manner minimising manual input, and information retrieval mechanisms. Techniques for the automatic filtering of data from flux tower and UAV sensors which will selectively decimate, compress and forward data dependent on the available bandwidth and the qualities of the data will be investigated. Techniques currently under development for the intelligent capture and transmission of weather station data (intended to optimize power usage and data quality) will be examined for suitability in the case of in-situ spectroradiometric measurements and UAVs data. Also the ability to automatically endow measurements with metadata concerning the types of equipment used and circumstances of data collection and subsequent automated integration of this data into SPECCHIO will both improve the validity of individual data sets and allow valid comparisons across data sets captured at different sites and using different equipment and regimes.
Spectroscopy Data Life Cycle in relation to the Spectral Information System
The following five activities will be conducted:
- Definition of a metadata set including a mandatory core set to document the particularities of ecophysiological spectral data. This metadata set will be aligned with general metadata sets of the in-situ spectroscopy community on an international level. The metadata set definition is expected to be an evolving standard based on best-practice throughout the duration of the Action.
- Definition and implementation of a dataflow from in-situ sensors to the spectral information system, submitting spectral data and available metadata in an automated fashion to allow for product generation downstream.
- Definition and implementation of data pre-processing and metadata augmentation including data flagging and filtering the support information building. This will also include the development on methodologies and according implementation for quality assessment and data check of ground spectral data by integrating metadata, auxiliary information and other radiation measurements available from existing eddy covariance databases.
- Development and implementation of information building functions that increase the information within the database by processing of available spectral data, metadata and auxiliary information where available.
- Implementation of data retrieval and spectral data assimilation into biogeochemical modeling, flux data spatialisation and linking space-sensed data with precise localised flux and spectral data.
The spectral information system implementing the above will be tested for some of the ICOS-reflectance sites to demonstrate its practical application and feasibility. The system will be based upon the latest version of the SPECCHIO spectral information system (Hueni, Nieke et al. 2009; Hueni, Malthus et al. 2011), which was recently redesigned and significantly enhanced by projects sponsored through the Australian National Data Service (ANDS) and the COST Action ES0903 (Hueni, Chisholm et al. 2012).
WG1 Deliverables:
- Definition of a mandatory metadata set, aligned with current international efforts in the spectroscopy community
- Develop an on-line instance of a spectral information database to serve as demonstration and testing platform for data sharing and information building
-> Public online database system: see www.specchio.ch
-> SPECCHIO API to program SPECCHIO from e.g. Matlab or R or Java
-> SPECCHIO online spectral database holding the data of OPTIMISE training schools (ABEL and SWAMP): SPECCHIO_Web_Interface - Develop a wireless automated dataflow from in-situ and UAV sensor for the database system
- Definition and implementation of data pre-processing and metadata augmentation algorithms and routines including quality checks and flagging and data assimilation
- Definition and implementation of system interfaces and algorithms for data retrieval allowing the building of products using sources such as biogeochemical modeling, flux data spatialisation and space-sensed data.
-> ASD iPad App to control an ASD field spectroradiometer, see also ASD iPad App Poster presented at EARSeL 2017, Zurich, Switzerland
-> Concept and Code for tower mounted instruments.
Data flow from tower mounted sensor to spectral database
-> Wireless User Interface for Flox/Rox point spectrometers: Open Source Code and compiled binaries are available on GitHub.
-> Matlab implementation of a pre-processing tool featured in an oral presentation at EARSeL: see EARSeL_Presentation
-> Matlab GUI interface linking Fluspect to SPECCHIO: see Fluspect-SPECCHIO GUI
-> Matlab Fluspect SPECCHIO GUI on Matlab Central: see Fluspect-SPECCHIO GUI Matlab Code
Interactive Matlab GUI interface linking Fluspect to SPECCHIO
Link to WG1 current members: WG1 Contact List April 17
