Quality Assurance

 

QUACK Indicator Checklist Guidance material
Input data
Scientific & methodological quality
Transparency Are all data sources referenced?
Do the metadata follow international standards?
our FAQ about metadata
Appropriateness Are the used data appropriate for the case study in terms of spatial and temporal scale?
Are the data the most up-to-date available?
Completeness Is the technical consistency checked (no outliers, no gaps in time series, etc.)?
Do you have all available data sets considered?
Quality assurance of data

Europe‘s 2016 indicator report

Processing
Scientific & methodological quality
Transparency Are the processing steps (i.e. scripts) well documented and reproducible?
Are the applied methods well documented and follow scientific standards (i.e. peer reviewed)?
Completeness Do the metadata of the produced data sets follow international standards? Metadata standards for climate impact indicators.
Ruth Petrie et al. 2016:
Appropriateness Are the used methods, for instance bias-adjustment, appropriate for the case study? Our FAQs and webinars
Accuracy/Robustness In case the show case is based on climate modelling data: Is the case study based on different emission scenarios, i.e. a low, medium and high scenario?
Is the case study based on different global or regional climate model simulations? In case that not all simulations of an ensemble are used, is the selection well explained and justified?
Our FAQs and webinars
How to use different RCPs

Value spread in Ensembles.

Guidelines for Use of Climate Scenarios Developed from Regional Climate Model Experiments
Mearns et al. 2003:

IPCC: Good Practice Guidance Paper on Assessing and Combining Multi Model Climate Projections
Knutti, et al 2010:

For regional indicators: Guidance for EURO-CORDEX climate projections data use(including validation,bias adjustment,robustness, etc.)

Validation Is there a procedure in place for a validation against independent data? our FAQs and webinars
Soil moisture compared to EO data

For regional indicators: Guidance for EURO-CORDEX climate projections data use (including validation, bias adjustment, robustness, etc.)
Published by the EURO-CORDEX community
euro-cordex-guidelines-version1.0-2017.08.pdf

Reflectivity Are the uncertainties assessed systematically in a standard manner?
Is the scientific consistency among multiple data sets and their findings well documented?
Statistical methods for the analysis of simulated and observed climate data - Applied in projects and institutions dealing with
climate change impact and adaptation
Hennemuth, et al. 2013 csc-report13_englisch_final-mit_umschlag.pdf
Practical relevance
Suitability for target group Is the relevance for the client assured?
Output
Scientific & methodological quality
Consistency Do the graphics and tables of the presentation match the captions and explanations? is coming
Transparency Are the produced data sets provided with metadata?
Are the limits of provided information disclosed?
Guidance for uncertainty assessment and communication
Arthur et al 2013
PBL_2013_Guidance-for-uncertainty-assessment-and-communication_712.pdf

PBL_2013_Guide-for-uncertainty-communication_1339.pdf

The Uncertainty Handbook: A practical guide for climate change communicators Adam Corner, Stephan Lewandowsky, Mary Phillips, Olga Roberts

Reliability Is the confidence of the data presented (e.g. the spread of the whole ensemble)? our FAQs and webinars

IPCC: Good Practice Guidance Paper on Assessing and Combining Multi Model Climate Projections Knutti, et al. 2010à

IPCC AR5 guidance note on consistent treatment of uncertainties: A common approach across the working groups Mastrandrea, et al , 2010:

Practical relevance
Lucidity Is the presentation clear and does it follow visualisation standards? is coming
Rights of use Are the rights of dissemination clarified (i.e. open access)? is coming
Usability Is the output intuitively presented? is coming
Usefulness Does the product help for problem solving? is coming
Outcome
Satisfaction
Usefulness Is the relevance for the client assured? is coming