Automation of Systematic Reviews of Diagnostic Test Accuracy

It is expensive and labour intensive to produce a systematic review - the highest quality of evidence in the medical sciences. Furthermore, clinical trials are being conducted at an accelerating rate. Methods to automate elements of review will enable decision makers to access the current evidence more rapidly. Existing methods are either coarse-grained and qualitative, or ad-hoc, and compromise too significantly in rigour to be trusted. By limiting the domain to reviews of diagnostic test accuracy, this project aims to formalise and leverage common features (such as cohort design, study structure, and performance metrics) to produce automated and reliable evidence summaries. This involves (1) automating the screening of evidence in both abstract and full text form, developing techniques to optimise the performance of large language models to a specific review question. In particular, this project is exploring the automated tuning of behaviour towards configurable characteristics, such as certain levels of recall and precision, which will enable different usage modes, such as a pre-screener for human review or an autonomous screening agent. (2) The extraction of quantitative evidence from clinical trial reports, enabling structured evidence summaries. This will focus on "living" systematic reviews, where the task is to update existing reviews given newly published trial reports; the existing review can function as specification for data extraction, and as a scaffold to structure and validate the data against. To this end, methods for parsing and modelling the structure of a trial from its report must be researched and developed. These methodologies together can then be used in case studies simulating the updating of existing reviews (for the evaluation of accuracy) as well as testing their effectiveness in supporting novel systematic reviews, to understand their implications in realistic scenarios. Areas: Artificial Intelligence Technologies; Information Systems.