A Spectrum of Research Software Quality
February 2020 (perspective of an assistant professor)
Research software varies widely in its quality or “realness.” Here's a spectrum from least to most “real.”
I've been building research software since my sophomore year of college when I joined my first lab; nearly all my publications are about research software or involve writing research software to do the underlying work. And in recent years, I've mostly been supervising students on building their own research software.
An age-old question that always comes up is, “How high-quality does my research software need to be?” Like many things in life, the answer is: it depends! Sometimes a hacked-together demo is all you need to demonstrate a proof-of-concept for a grant proposal or academic paper; other times your software must be good enough to deploy to users to measure real-world usage.
Anyhow, here's my quick attempt at a spectrum of research software quality, in ascending order:
Appendix: related links
Thanks to Jonathan Edwards and John Regehr for feedback.