Photo courtesy of the Hong Kong Polytechnic University
Researchers from Hong Kong are developing a clinical-grade ophthalmic AI co-pilot system that not only assists doctors in diagnosing diseases but also supports treatment planning and follow-up management.
The team at Hong Kong Polytechnic University (PolyU) have developed a prototype AI-enabled system – dubbed EyeAgent 1.0 – based on multimodal data, including clinical texts and images, that reportedly can identify about 260 ophthalmic conditions.
Following pilots across the city and mainland China, the AI system is now being further developed under the EyeAgent 2.0 project, with advanced clinical reasoning and disease trajectory modelling capabilities.
WHAT'S IT ABOUT
In an interview with Mobihealth News, research lead Dr He Mingguang revealed that the second version is being built on a large-scale, real-world longitudinal ophthalmic dataset spanning Hong Kong, mainland China, and India.
"Through collaborations with a network of more than 30 eye centres and hospitals, we are aggregating and standardising over one million electronic medical records and imaging datasets," said the Henry G. Leong Professor in Elderly Vision Health at the PolyU School of Optometry. Data will include fundus photography, OCT/OCTA, slit-lamp imaging, visual field data, clinical text, and structured examination records.
EyeAgent 1.0 was trained on over 2.7 million ophthalmic images and 14 specialist textbooks, processing 23 data modalities to identify hundreds of eye conditions. It incorporated more than 50 ophthalmology AI tools developed by the same team.
To address data heterogeneity, annotation inconsistency and cross-site variability, the team is implementing structured data cleaning, temporal alignment, unified patient representation and cross-centre harmonisation, according to Dr Jing Zhang, a research assistant professor at the PolyU School of Optometry.
On the modelling side, the team is developing a contrastive self-supervised learning framework based on transformer architectures to align image, text and structured clinical data within a shared representation space. That includes optimised vision transformers and language encoders for both cross-modal representation learning and temporal disease-progression modelling, Dr Zhang said.
The team will pretrain the self-supervised foundation model, "with a target of AUC ≥ 0.90 on benchmark diagnostic tasks."
Meanwhile, the second layer – a multi-agent clinical reasoning engine for diagnosis, treatment planning and follow-up – will be validated through a clinical simulation platform, with a target of ≥85% reasoning fidelity, Dr Zhang added.
WHY IT MATTERS
Amid ageing populations and a shortage of specialists, improving the consistency and efficiency of ophthalmic diagnosis remains a global challenge, which the PolyU team aims to address with EyeAgent 1.0.
Based on prior validation against human assessments, the prototype reduced response time by 56.8%, improved diagnosis rate by 24.5%, and achieved 89.9% user satisfaction, Prof He shared.
But it had an important technical limitation, according to the research lead. "It was built primarily on cross-sectional data, meaning it was strong at supporting diagnosis at a single visit, but not designed to model disease progression, treatment response or longitudinal risk over time."
The team expects the upgraded ophthalmic AI co-pilot system – leveraging more than one million longitudinal patient records – to "support the full clinical reasoning pathway, from diagnosis and prognosis to treatment planning, follow-up, and dynamic adjustment over time," he said.
Their goal is to develop EyeAgent 2.0 into a software as a medical device compliant with regulatory standards, with plans to progressively advance registration and deployment efforts following completion of clinical validation. The team is targeting completion within two years, supported by government funding.
"EyeAgent 2.0 will first be deployed in three or more clinical sites through our Hong Kong partner hospital network, where we will assess diagnostic concordance, decision-time reduction and clinician satisfaction," Prof He told this publication. A cross-region validation will follow.
"In parallel, we will complete the risk analysis, validation, and technical documentation required for SaMD readiness."
A hybrid business model combining annual subscriptions with usage-based charges is also envisioned, with flexible deployment tailored to different hospital information systems.
THE LARGER TREND
A similar foundation AI model has been developed at the Chinese University of Hong Kong to automate eye disease diagnosis. The VisionFM model was pre-trained on a large ophthalmic dataset containing 3.4 million images across eight eye imaging modalities. It demonstrated diagnostic performance comparable to that of an ophthalmologist with four to eight years of clinical experience.
Across the Asia-Pacific, AI models have also been developed to analyse eye images for a range of conditions, including attention deficit hyperactivity disorder, ultraviolet damage-related skin cancer, Alzheimer's disease, and kidney disease. Another study in Australia screened 50,000 retinal images using AI to identify links between the retina and diseases, such as neurodegenerative and cardiometabolic disorders.
