Transparent reporting
Once you’ve performed your analysis, it is important to report your research in a transparent way. This allows others to fully judge the steps taken and the results obtained.
Reporting guidelines are widely used, and increasingly expected by funders and publishers. Reporting guidelines list all aspects of a study which need to be reported. The
EQUATOR Network (Enhancing the QUAlity and
Transparency Of health Research) brings together reporting guidelines for different study types.
For observational studies this includes the
STROBE guidelines and
TRIPOD+AI guidelines.
While there are difference across the guidelines, common elements include:
- The source of the data
- How the final sample size was arrived at
- Definitions of different types of variables included in the analysis
- How missing data were addressed
- Descriptive statistics of the sample
- Results from the analyses
- Limitations of the study
- Interpretation of the results
- Funding and the role of the funder
Reporting of machine learning studies
There are particular concerns with reporting standards for machine learning based prediction studies (see, for example, "Reporting quality of studies using machine learning models for medical diagnosis: a systematic review" or "Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review"
Some specific concerns include:
- The ‘black-box’ nature of many algorithms make them hard to reproduce without suitable reporting of e.g. hyperparameters chosen
- Model training and development can be a stochastic process
- The intersection of statistical and computer science traditions can result in inconsistent language
To that end, the
TRIPOD+AI reporting checklist has been developed for the reporting of clinical prediction models which use machine learning (and/or regression).
Additionally,
TRIPOD-LLM is the reporting guideline for those studies which use large langauge models