Upper limb rehabilitation after cervical spinal cord injury is vital for regaining independence. Hand function assessments are critical for upper limb rehabilitation, but are unable to accurately reflect performance in the individual’s home environment. Video from wearable cameras (egocentric video), paired with deep learning, can assess hand function during activities of daily living (ADLs) at home. Previous studies have focused on analyzing quantitative aspects of hand usage, but there has yet to be a study that uses deep neural networks to assess the quality of hand usage from egocentric video.
To train a deep neural network to predict hand function assessment scores from egocentric video.
The dataset used contained egocentric videos of ADLs performed by 17 participants with AIS grades from A-D in a home simulation laboratory. Tasks were annotated with scores adapted from the Graded Redefined Assessment of Strength Sensibility and Prehension (GRASSP). The annotated video was then used to train and validate a SlowFast neural network to predict GRASSP scores, using leave-one-subject-out cross validation. Model performance was evaluated by mean absolute error, accuracy, and F1 score. The model was optimized with a hyperparameter sweep.
The top performing model demonstrated a mean absolute error of 0.52±0.19, an accuracy of 0.55±0.14, and F1 score of 0.55±0.16, on an ordinal scale from 1 to 5.
These results demonstrate that automated assessment of hand function is possible by applying deep learning to egocentric video. Future work should expand the model to larger datasets with more variability.
