[en] In this paper, we explore the concept of providing feedback to a user moving in front of a depth camera so that he is able to replicate a specific template action. This can be used as a home based rehabilitation system for stroke survivors, where the objective is for patients to practice and improve their daily life activities. Patients are guided in how to correctly perform an action by following feedback proposals. These proposals are presented in a human interpretable way. In order to align an action that was performed with the template action, we explore two different approaches, namely, Subsequence Dynamic Time Warping and Temporal Commonality Discovery. The first method aims to find the temporal alignment and the second one discovers the interval of the subsequence that shares similar content, after which standard Dynamic Time Warping can be used for the temporal alignment. Then, feedback proposals can be provided in order to correct the user with respect to the template action. Experimental results show that both methods have similar accuracy rate and the computational time is a decisive factor, where Subsequence Dynamic Time Warping achieves faster results.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > SIGCOM
Disciplines :
Computer science
Author, co-author :
Baptista, Renato ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Aouada, Djamila ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Ottersten, Björn ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
External co-authors :
no
Language :
English
Title :
Video-Based Feedback for Assisting Physical Activity
Publication date :
2017
Event name :
12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP)
Event date :
February 27-March 01, 2017
Main work title :
12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP)
Peer reviewed :
Peer reviewed
European Projects :
H2020 - 689947 - STARR - Decision SupporT and self-mAnagement system for stRoke survivoRs
FnR Project :
FNR10415355 - 3d Action Recognition Using Refinement And Invariance Strategies For Reliable Surveillance, 2015 (01/06/2016-31/05/2019) - Bjorn Ottersten
Antunes, M., Aouada, D., and Ottersten, B. (2016a). A revisit to human action recognition from depth sequences: Guided svm-sampling for joint selection. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1-8.
Antunes, M., Baptista, R., Demisse, G., Aouada, D., and Ottersten, B. (2016b). Visual and human-interpretable feedback for assisting physical activity. In European Conference on Computer Vision (ECCV)Workshop on Assistive Computer Vision and Robotics Amsterdam,.
Burke, J. W., McNeill, M., Charles, D., Morrow, P., Crosbie, J., and McDonough, S. (2009). Serious games for upper limb rehabilitation following stroke. In Proceedings of the 2009 Conference in Games and Virtual Worlds for Serious Applications, VS-GAMES '09, pages 103-110, Washington, DC, USA. IEEE Computer Society.
Chaaraoui, A. A., Climent-Pérez, P., and Flórez-Revuelta, F. (2012). A review on vision techniques applied to human behaviour analysis for ambient-assisted living. Expert Systems with Applications.
Chu, W.-S., Zhou, F., and De la Torre, F. (2012). Unsupervised temporal commonality discovery. In ECCV.
Gupta, A., He, J., Martinez, J., Little, J. J., and Woodham, R. J. (2016). Efficient video-based retrieval of human motion with flexible alignment. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).
Hondori, H. M., Khademi, M., Dodakian, L., Cramer, S. C., and Lopes, C. V. (2013). A spatial augmented reality rehab system for post-stroke hand rehabilitation. In MMVR.
Kato, P. M. (2010). Video Games in Health Care: Closing the Gap. Review of General Psychology, 14:113-121.
Kulkarni, K., Evangelidis, G., Cech, J., and Horaud, R. (2015). Continuous action recognition based on sequence alignment. International Journal of Computer Vision, 112(1):90-114.
Kwakkel, G., Kollen, B. J., and Krebs, H. I. (2007). Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabilitation and neural repair.
Langhorne, P., Taylor, G., Murray, G., Dennis, M., Anderson, C., Bautz-Holter, E., Dey, P., Indredavik, B., Mayo, N., Power, M., et al. (2005). Early supported discharge services for stroke patients: a meta-analysis of individual patients' data. The Lancet.
Mousavi Hondori, H. and Khademi, M. (2014). A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. Journal of Medical Engineering, 2014.
Müller, M. (2007). Dynamic Time Warping. Springer.
Ofli, F., Kurillo, G., Obdrzálek, S., Bajcsy, R., Jimison, H. B., and Pavel, M. (2016). Design and evaluation of an interactive exercise coaching system for older adults: Lessons learned. IEEE J. Biomedical and Health Informatics.
Pirsiavash, H., Vondrick, C., and Torralba, A. (2014). Assessing the quality of actions. In Computer Vision-ECCV 2014, pages 556-571. Springer.
Rakthanmanon, T., Campana, B., Mueen, A., Batista, G., Westover, B., Zhu, Q., Zakaria, J., and Keogh, E. (2012). Searching and mining trillions of time series subsequences under dynamic time warping. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '12, pages 262-270, New York, NY, USA. ACM.
Sucar, L. E., Luis, R., Leder, R., Hernandez, J., and Sanchez, I. (2010). Gesture therapy: a visionbased system for upper extremity stroke rehabilitation. In Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE.
Sun, F., Norman, I. J., and While, A. E. (2013). Physical activity in older people: a systematic review. BMC Public Health.
Tao, L., Paiement, A., Damen, D., Mirmehdi, M., Hannuna, S., Camplani, M., Burghardt, T., and Craddock, I. (2016). A comparative study of pose representation and dynamics modelling for online motion quality assessment. Computer Vision and Image Understanding.
Veerbeek, J. M., van Wegen, E., van Peppen, R., van der Wees, P. J., Hendriks, E., Rietberg, M., and Kwakkel, G. (2014). What is the evidence for physical therapy poststroke? a systematic review and meta-analysis. PloS one.
Vemulapalli, R., Arrate, F., and Chellappa, R. (2014). Human action recognition by representing 3d skeletons as points in a lie group. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Wang, R., Medioni, G., Winstein, C., and Blanco, C. (2013). Home monitoring musculo-skeletal disorders with a single 3d sensor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 521-528.
Xia, L., Chen, C., and Aggarwal, J. (2012). View invariant human action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pages 20-27. IEEE.
Zhou, F. and Torre, F. (2009). Canonical time warping for alignment of human behavior. In Bengio, Y., Schuurmans, D., Lafferty, J. D., Williams, C. K. I., and Culotta, A., editors, Advances in Neural Information Processing Systems 22, pages 2286-2294. Curran Associates, Inc.
Zhou, H. and Hu, H. (2008). Human motion tracking for rehabilitationa survey. Biomedical Signal Processing and Control, 3(1):1-18.
