- Clinical Measurement and Development
- Carly Booker
- Training location
- South London Consortium
Objectives: The aim of this project was to produce an expert system using machine learning which writes instrumented gait analysis reports for children with cerebral palsy based on the ‘opinion-evidence’ model of reporting, with an accuracy of 90% when compared to the gold standard of expert clinician reporting. Methods: The One Small Step Gait Laboratory has a bank of reports from previous assessments which use the ‘opinion-evidence’ model to report findings from instrumented gait analysis. The opinions and evidence from reports written about all individuals with a diagnosis of cerebral palsy who underwent instrumented gait analysis in 2016 were extracted and used to train a neural network. Evidence and opinions were split into three groups: strongly present, mildly present and not present. The evidence from a random sample of assessments from 2017 was put through the model and the output of the neural network was compared to the actual findings reported by expert clinicians in the 2017 reports to measure accuracy. Some preliminary work was completed to optimise the design of the neural network, including the number of hidden layers used and the size of the hidden layers. Results: The neural network had an accuracy of 72% with limited clinical usefulness due to the low sensitivity of predictions. Data sparsity due to large numbers of possible items of supporting evidence is a strong contributor to the poor accuracy and sensitivity. Simplifying the dataset to split the evidence and opinions into two groups (present and not present) was moderately more successful, with an accuracy of 85% although still not reaching the desired accuracy of the model. Splitting the evidence into groups for each clinical opinion and creating separate neural networks for individual clinical opinions produced accuracies of 90% and greater when used with the simplified dataset. Conclusions: Neural networks may not be the best approach for an expert system for instrumented gait analysis reports due to large numbers of possible items of supporting evidence and clinical opinions. Further work is needed on separate network models for each clinical opinion to investigate the potential. Other machine learning methods which are less sensitive to data sparsity should be also be investigated.