A research team from the University of Illinois Urbana-Champaign has developed a new method for assessing clinical trial quality using artificial intelligence. The study trains a natural language processing model to automatically detect whether medical research reports comply with CONSORT standards, providing technical support to improve the quality of clinical trial reporting.

Randomized controlled clinical trials are essential for evaluating the safety and efficacy of new therapies. However, incomplete reporting of trial details in existing medical literature often hinders accurate assessment of research quality. The Illinois team utilized the Bridges-2 supercomputing system at the Pittsburgh Supercomputing Center (PSC) to develop an AI tool capable of identifying missing information in clinical trial reports.

Associate Professor of Information Sciences Halil Kilicoglu stated: "Clinical trials are considered the best evidence for clinical treatments. To use a drug for a disease, we need to prove it is safe and effective. But publications of clinical trials have many issues—they often lack sufficient detail and transparency about procedures, making it difficult to assess the rigor of the evidence."

The research team constructed an AI training dataset based on 83 recommendations from the CONSORT 2010 and SPIRIT 2013 statements. They collected 200 clinical trial articles published between 2011 and 2022 and trained a Transformer neural network model using the GPU units of the Bridges-2 system. This AI clinical trial evaluation tool learns to recognize characteristics of standards-compliant research reports by analyzing text patterns.

In model testing, the best-performing natural language processing program achieved an F₁ score of 0.742 at the sentence level and 0.865 at the full-text level. The researchers published these preliminary results in Scientific Data in February 2025. Kilicoglu noted: "We are developing deep learning models that require GPUs. After registering with Bridges, we can access GPU resources, and the necessary software is usually pre-installed, which greatly facilitates our work."

The team plans to further optimize the AI clinical trial evaluation tool by increasing training data and applying model distillation techniques to improve performance. The ultimate goal is to develop an open-source tool for use by research authors and academic journals to enhance the design, implementation, and reporting quality of clinical trials. This AI detection method based on CONSORT standards is expected to promote standardization in medical research.