AI Advances Could Revolutionize Support for Childhood Cancer Survivors

Artificial intelligence (AI) holds significant potential to assist physicians in determining if childhood cancer survivors require additional support. New research from St. Jude Children's Research Hospital, published in Communications Medicine, highlights a crucial finding: AI performance significantly improves with the inclusion of more information in its prompts.

Scientists observed large language models (LLMs) as they analyzed interviews with young survivors and their caregivers. The primary objective was to detect multiple symptoms that cause significant disruptions in daily life. Different prompting approaches were rigorously compared, revealing a clear trend:

More complex prompts, which provided additional information to the models, consistently yielded the best results.

This compelling discovery suggests that future applications of AI aimed at enhancing survivor care should prioritize sophisticated prompting strategies.

Expert Insights on Conversational Data

Dr. I-Chan Huang, corresponding author from St. Jude Department of Epidemiology & Cancer Control, emphasized the significance of this work. "Forty percent to 60% of clinical encounters involve patient-physician discussions about symptoms," Dr. Huang stated. "The study provides a proof of concept that LLMs could analyze this conversational data to identify symptom severity and functional impact, thereby supporting physician decision-making."

Addressing the Challenge of Long-Term Care

Childhood cancer treatment, while life-saving, can unfortunately lead to a range of long-term health issues. Identifying survivors with symptoms severe enough to warrant targeted support has traditionally been a challenging endeavor. This process often relies on valuable, but difficult-to-review, data from conversational transcripts and open-ended survey responses. Language-based AI offers a promising new method to efficiently analyze this critical information.

The St. Jude Study: Methodology and Findings

The research involved interviews with 30 survivors, aged 8 to 17, and their caregivers. Two human experts meticulously analyzed conversation transcripts for excessive pain and fatigue. They categorized symptoms by severity and their physical, cognitive, or social impact, generating over 800 analyzable data points.

The same transcripts were then given to leading AI models, ChatGPT and Llama, using four distinct prompting styles:

• Zero-shot
• Few-shot (simple)
• Chain-of-thought
• Generated knowledge (complex)

The study found that simple prompts were largely ineffective. However, the complex prompting strategies—chain-of-thought and generated knowledge—performed significantly better and showed higher concurrence with human reviewers. Both complex methods proved highly effective at identifying the physical and cognitive impact of symptoms, with a moderate ability to detect social impacts.

Paving the Way for Future AI Applications

While extensive testing is still needed before clinical integration, these initial findings are highly encouraging. They strongly suggest that chain-of-thought, generated knowledge, or similar complex prompting methods should be prioritized in future applications. This study provides an early, yet powerful, example of how AI might significantly enhance survivorship care by making complex symptom information from patient-physician conversations more accessible and analyzable.