Name of Project: VisioTherapy: Building an exercise quality dataset using a community of physiotherapists at professional rugby and sports clubs
Team Website (if applicable): www.visiotherapy.ai (coming soon)
Proposal Wallet Address (*mandatory): 0x36f741F4808a329C9E876F551Bcf337B7dDc54Ff
The proposal in one sentence: We are on-boarding a community of physiotherapists at professional sports clubs to Ocean by crowdsourcing an exercise quality dataset of strength and rehab exercises labelled using a mobile app
Which category best describes your project? Pick one or more.
- [x] Build / improve applications or integrations to Ocean
- [x] Outreach / community / spread awareness (grants don’t need to be technical in nature)
- [x] Unleash data
Funding Amount: Enter the amount of OCEAN your team is requesting (limit 18.000 OCEAN) 27.200 OCEAN
- Description of the project:
The Sports Analytics market is expected to reach USD 5.11 Billion annually by 2026, the Sports Medicine market is estimated at over USD 5 billion annually and the global market for athlete tracking systems was USD 2.26 billion in 2018. Sports participation statistics show the following:
|Sport||Elite & High Performance||Youth & Grassroots|
|Rugby||40,000+ (Pro & HP)||11 million|
At the same time, telehealth is quickly becoming more popular in the delivery of physiotherapy strength and rehabilitation exercises. Remote delivery of coaching and rehab reduces pressure on the health system, and facilitates access to non-critical services during emergencies such as pandemics. It also facilitates wider access to services where the physio may be far away.
Deep learning models for computer vision have the potential to assist physios in video analysis. Action quality assessment (AQA) involves the quantification of how well actions are carried out. 2D and 3D human pose estimation (HPE) helps to identify the position of the skeleton subject. 3D shape estimation estimates the surface mesh of the subject. 3D force estimation can predict the ground reaction forces at each foot, and the torques at each joint. Our start-up team at VideoForce that includes physiotherapists, biomechanical engineers and computer scientists have been collecting a dataset of videos from physiotherapists and trial partners at a number of high-profile sports clubs (see Prior Work section). An example frame from a video is shown below. With labels provided by the physiotherapists, we can incorporate domain knowledge into the models.
The purpose of this proposal is to create a decentralised video dataset of physiotherapy strength and rehabilitation exercises by creating an app (building off the open source app of DataUnion) where an existing community of physiotherapists can record videos and add skeleton labels. The resulting dataset will be owned by the individuals that contribute to it. The contributions and ownership of the data will be handled in a data union. We are collaborating with DataUnion to make this happen.
- What problem is your project solving?
This project builds on the incentive and ownership infrastructure of co-owned datasets on Ocean Protocol, and the suite of data labelling tools of DataUnion to onboard a new community of physios and sports clubs to the Ocean ecosystem, as shown in the figure below. This solves problems related to a centralized Web 2 approach by (i) improving incentives for physios to provide domain knowledge to be used for building automated video assessment tools, (ii) providing ownership of a newly-created video dataset to the contributors, and (iii) crowdsourcing data labelling by physios using a new application that integrates with Ocean Protocol.
- Improving incentives by physios to input domain knowledge to the system
The physios have weak incentives to provide value - in the form of domain knowledge and time - by uploading videos and adding labels based on their own experience and domain knowledge. Currently, they are incentivised to contribute by on the results that we return to them from our suite of models. At the moment, we use hand-crafted features for this based on the skeleton predicted by 3D HPE models, rather than deep learning models for action quality assessment. This is a chicken-and-egg scenario in that we need the data to train accurate models, while we need accurate models to bring in data.
The contributors should be rewarded by ownership of the resulting dataset, more control over what it is used for and the ability to earn from models that are trained on the data. This project will create a proof-of-concept of a new incentive structure for crowdsourcing labels from physios. This will activate a new community into the data economy. The value proposition and market feasibility of this new approach will be further developed and evaluated, and a number of interviews will be performed with stakeholders.
- Providing ownership of data to physios and sports clubs
There is little infrastructure in place to provide data ownership to the physiotherapists and sports clubs that collaborate with us (a sports club and a head physiotherapist are shareholders in VideoForce but this is not easily scaled to more clubs and physios). Furthermore, the current state of ethics applications in companies and universities is not satisfactory to ensure control of individuals over data. The procedures in place are also slow.
This project will form a data union around this type of data and community. Some funding will be used to further develop a circular business model to scale the ecosystem safely and fairly, and for unlocking customers of associated data and models.
- Improving the uploading and labelling process with an app
The current approach of uploading and labelling videos is insecure and cumbersome. Tools for uploading and labelling videos will help to scale the process. Physiotherapists will log into the app and a digital wallet will be created for them. The users will upload videos and add annotations using a binary label to indicate whether the action was performed with good or bad quality. More sophisticated future models could have multi-class labels, indicating the attributes of the exercise that suggest good or bad quality and possibly give advice on how to improve the execution. The labeller will then verify that the data and annotations are correct and will be rewarded with shares of the dataset that they contributed to through tokens.
- What is the final product (e.g. App, URL, Medium, etc)?
A web-based application or mobile app for labelling videos of physiotherapy exercises.
- How does this project drive value to the Ocean ecosystem?
Firstly, the project will build an app to introduce a new community of physiotherapists to the ecosystem. These individuals may be attracted by the core values of privacy and ownership of data, and become stakeholders as a result. There were around 560.000 practising physiotherapists in the EU-27 in 2018. The app will onboard physios and create a digital wallet. We assume 0.01% of them (56 individuals) invests in OCEAN tokens as a result of this project with an average investment of 1000 $OCEAN. This results in a potential of a Total Value Locked (TVL) of 560.000 * 0.01% * 1000 $OCEAN = 56.000 $OCEAN demand.
Secondly, the project may introduce sports clubs to the ecosystem as new data publishers, who can be expected to have a higher stake than the average physiotherapist user. We assume that 1 sports club invests 30.000 $OCEAN, giving a TVL of 30.000 * 1 = 30.000 $OCEAN.
Thirdly, the project will create a unique new dataset related to health and sports that may attract a new type of investor in data tokens. We assume that an average high quality data pool has a TVL of 100.000 $OCEAN over one year. For a single initial dataset (combined across sports clubs), we get a TVL in data pools of 100.000 * 1 = 100.000 $OCEAN.
bang = 56.000 + 30.000 + 100.000 = 186.000 $OCEAN
buck = 18.000 $OCEAN
(% chance of success) = 80%
ROI = 186.000 $OCEAN / 18.000 $OCEAN * 0.8 = 8.2
This is above the expected ROI of 1.0.
Project Deliverables – Category
IF: Build / improve applications or integration to Ocean, then:
- App will be live, at: www.visiotherapy.ai or iOS AppStore + GooglePlay Store (url or app store)
- Software will be open-source with a permissive license based on existing work by DataUnion
IF: Outreach / community, then (one or more of):
- Blog posts will be published e.g. at medium.com, techireland.org, siliconrepublic.com - illustrating the community opportunities to increase activation
- A video production will be published at youtube.com, possibly including professional athletes at elite sports clubs in our community
- Outline possible circular business models, based on decentral set-up, and a respective roadmap
IF: Unleash data, then:
- Data will be made available on Ocean Market via compute-to-data
Project Deliverables - Roadmap
- Any prior work completed thus far?
Our team includes physiotherapists, biomechanical engineers, computer scientists and business developers. Work has been ongoing since January 2020 in university (with limited funding), with development focusing on validation of publicly-available models and research into novel model architectures. We have also established relationships with a large community of physiotherapists and sports clubs. Leinster Rugby (4 times European champions) are a close research partner and a shareholder in the company. The Irish Rugby Football Union have recently approved access to the international team (4 times Six Nations Championship winners) and the remaining 3 provincial Irish clubs and are including us in a small group of innovative sports tech start-ups to work with. We have also completed a market feasibility study (worth ~€20k funding) with 60 individuals and companies identified as potential customers and stakeholders. To date, 25 of these have been approached with interviews, conversations, presentations, and online sessions. A number of these have signed agreements to be trial partners for our concierge MVP, which provides video-analysis-as-a-service on videos of athletes performing squats. The trial partners to date are:
- Leinster Rugby
- Irish Rugby Football Union
- Sydney Swans (5 times Australian Football League winners)
- Emovi (bio-medical device company for knee joint assessment)
- Irish Rowing
In some cases where an official trial partnership is a more in-depth process physiotherapists were still willing to provide input on an individual basis:
- Arsenal FC
- Manchester City FC
- Fulham FC
- Scottish FA
- Redbull Innovation Centre
- Badminton England
The clubs have also indicated their interest in obtaining video analysis of training and matches from cameras in training pitches and stadiums. This touches on a new market of data and models for the sports analytics community that can be reached in future. Several other use cases emerged from the study:
- Elite Athlete market for both coaches and physios (across multiple sports)
- Elite Performer market for coaches and physios (e.g., Cirque du Soleil, Disney, Ballet etc.)
- Youth and development athlete markets in sport and performance (Coaching & Rehab)
- Grassroots sports and performance markets (Coaching & Rehab)
- Consumer market (e.g., At home Physio consultations, medical consultations, healthcare industry generally)
- At home training market (e.g., Yoga, Pilates, Mirror, Tonal, Wattbike etc.)
- Workplace training (Health and Safety) market
- Sports (TV) analysis (e.g., along the lines of Top Tracer for ball trajectory but for body motion, force etc.)
- Other market opportunities also include –Teaching, the possibility to provide APIs or SDK for 3rd party applications and partners to leverage the technology.
Overall, the feedback overwhelmingly indicated that existing tools for providing assistance to physios are not used due to limited accuracy and time-consuming manual inputs required. Automated tools for video analysis using deep learning models on domain-specific datasets have the potential to revolutionise physiotherapy treatment.
- What is the project roadmap? That is: what are key milestones, and the target date for each milestone.
Jun 1 – 7: Design procedure for video labelling, in terms of number of labellers per video, verification etc.
Jun 7 – Jun 30: Work on feature for onboarding physios in app using digital wallets
Jul 1 – Jul 31: Work on feature for uploading videos
Aug 1– Aug 30: Work on features for annotation and validation of the annotation.
Business Model Development
Jun 1 – Jul 31: Develop a circular business model to unlock data and customers, and provide value to the community.
- Please include the milestone: publish an article/tutorial explaining your project as part of the grant (eg medium, etc).
Blog posts will be published e.g. at medium.com, techireland.org, siliconrepublic.com. These will be targeted especially at the physiotherapy community and potential customers such as sports clubs. Tutorials for uploading/labelling images will also be published.
- Please include the team’s future plans and intentions.
With a dataset of exercise quality created, we are going to train a set of models on this data and offer the use of the algorithm in the app to enable advanced assessment capabilities by users on 2D videos. This is a proof-of-concept for tele-physio services. Potential customers include less-experienced physiotherapists, strength and conditioning coaches, personal trainers or amateur clubs and athletes who want access to models that have built-in domain knowledge from physios at professional sports clubs. The model will also be sold via a marketplace on Ocean Protocol.
We will also expand the models on offer in the Ocean Protocol marketplace to include 2D and 3D human pose estimation (HPE). Human pose estimation is used to predict the joint positions of a human from images or video. These models will be useful tools to help physios assess exercise quality as both data labellers and customers - the algorithms will create a 3D skeleton on top of the 2D images.
To build a training dataset for 2D HPE, features will be added to the mobile app to annotate the skeleton of subjects in videos of exercises. 3D HPE models require motion capture datasets. We have close relationships with a number of universities, sports clubs and hospitals that have large MoCap datasets of exercises for physiotherapy applications. While generally happy to share, there are a number of hurdles that slow down the process. Firstly, ethical approval needs to be sought from a panel to share the data with each new group of researchers. Secondly, licensing arrangements need to be discussed every time with each new customer. These can take a long time. Also, there is no infrastructure for compute-to-data or federated learning in the biomechanics community. The data is thus duplicated on the servers of new research groups, which increases security risk. There is huge potential and upside to introduce Ocean Protocol to this ecosystem and our success story will pave the way to do so.
For each team member, give their name, role and background such as the following.
The team members are listed in the order of them joining the project.
Dr. Richard Blythman
- Role: machine learning engineer, biomechanical engineer, sports enthusiast
- Relevant Credentials (e.g.):
- Founder VideoForce
- Video Intelligence Researcher at Huawei Technologies
- Research Fellow (Computer Science), Trinity College Dublin
- Machine Learning R&D Engineer at FotoNation, Xperi
Dr. Mark Siebert
- Data publishing (10yrs)
- Business Development for Data Markets
- Owning and driving global executive engagements and partnerships, Data and Open Science
- Web3 experience: < 1 Year
- Positioning businesses in emerging markets or innovative fields with focus on data and AI-driven solutions