An eScience perspective on Replicability and Reproducibility

ResearchSpace/Manakin Repository

Show simple item record Gahegan, Mark en
dc.coverage.spatial Tempe, Arizona, USA en 2020-02-11T21:21:15Z en 2019-02-12 en
dc.identifier.citation Replicability and Reproducibility in Geospatial Research: A SPARC Workshop, 11 Feb 2019 en
dc.identifier.uri en
dc.description.abstract Replicability and Reproducibility in Science, and the roles played by web semantics. Reproducibility in science has become a very visible issue of late, with several key papers pointing out significant issues in reproducing highly cited research in the fields of psychology and bioinformatics. The movement towards in-silico science (simulation and modelling) means that some experiments are ‘born digital’ and thus should have a valid digital representation by default (though not necessarily a complete one). Other forms of science still require laboratory work (in-vitrio), yet even here, reproducibility is making ground. Advanced computational workflows and scientific notebooks are now shrinking the reproducibility gap, though finally closing it represents a big challenge. Web semantics have some key roles to play in supporting truly open and reproducible science, and to this end we here (i) provide a taxonomy of approaches to reproducibility, (ii) show how semantic web technologies help to empower them, and (iii) describe some of the as yet unaddressed challenges. Reusability and replication of analytical tasks and experiments can be supported at many levels. Throughout this paper we adopt an approach to open, reproducible science drawn from… • A model is replicable when re-running the source code produces a consistent result. In this case, literally a digital replica of the original experiment produces the same answers. • A model is reproducible when its outputs can be reproduced by a machine from an unambiguous statement of the model equations, together with specified values of the model parameters, initial conditions and boundary conditions. In other words, the model can move from its originating source code implementation, and by means of an underlying representation based on domain theory (mathematics, logic or a mix of both) it can be successfully reproduced in some new system. • A model is reusable when it can be used independently or as a module within another model. This requires that the model is well documented, the source code is available and that its limitations and appropriate use are clear. One way to describe these model features is by semantic annotation, which can provide unambiguous meaning to the variables, parameters and control sequences used. This kind of annotation requires detailed model semantics to be developed and adopted by a research community. Many research communities aspire to these kind of model semantics, which are often more difficult to capture than data semantics (and those are hard enough) • A model is discoverable when it has been annotated with metadata that describe the purpose and use of the model sufficiently to allow it to be found and accessed via a webservice. Exactly what kinds of metadata are needed here is an open question. Certainly, details of the meaning of the model (see reusable) can be helpful, as can example use-cases, user feedback, statements about accuracy and some idea of the context surrounding the research task. Gahegan & Adams (2014) have some further ideas about how such semantics might be uncovered and used. • A model is validated when its predictions under specified conditions match experimental observations. In other words, validation requires that we test a model against real-world observations, not just for consistency within own internal logic or mathematics. Models are typically validated within a range of ‘safe’ operating conditions (such as a scale interval, or between two temperature values). en
dc.relation.ispartof Replicability and Reproducibility in Geospatial Research: A SPARC Workshop en
dc.rights Items in ResearchSpace are protected by copyright, with all rights reserved, unless otherwise indicated. Previously published items are made available in accordance with the copyright policy of the publisher. en
dc.rights.uri en
dc.title An eScience perspective on Replicability and Reproducibility en
dc.type Presentation en
dc.identifier.doi 10.17605/OSF.IO/GVP3Q en
dc.rights.holder Copyright: The author en en
pubs.finish-date 2019-02-12 en
pubs.start-date 2019-02-11 en
dc.rights.accessrights en
pubs.subtype Conference Oral Presentation en
pubs.elements-id 790527 en Science en School of Computer Science en
pubs.record-created-at-source-date 2020-01-07 en

Full text options

Find Full text

This item appears in the following Collection(s)

Show simple item record


Search ResearchSpace

Advanced Search