-
Name of Project: VisioTherapy: Building an exercise quality dataset using a community of physiotherapists at professional rugby and sports clubs
-
Team Website (if applicable): www.visiotherapy.ai (coming soon)
-
Proposal Wallet Address (*mandatory): 0x36f741F4808a329C9E876F551Bcf337B7dDc54Ff
-
The proposal in one sentence: We are on-boarding a community of physiotherapists at professional sports clubs to Ocean by crowdsourcing an exercise quality dataset of strength and rehab exercises labelled using a mobile app
-
Which category best describes your project? Pick one or more.
- [x] Build / improve applications or integrations to Ocean
- [x] Outreach / community / spread awareness (grants don’t need to be technical in nature)
- [x] Unleash data
-
Funding Amount: Enter the amount of OCEAN your team is requesting (limit 18.000 OCEAN) 27.200 OCEAN
Project Overview
- Description of the project:
The global market for sports analytics is forecasted to reach $5.11B per year by 2026. The market for sports medicine is approximated at $5B+ per year. The athlete tracking systems market in 2018 was estimated at $2.26B.
At the same time, telehealth is quickly becoming more popular in the delivery of physiotherapy strength and rehabilitation exercises. Remote delivery of coaching and rehab reduces pressure on the health system, and facilitates access to non-critical services during emergencies such as pandemics. It also facilitates wider access to services where the physio may be far away.
Deep learning models for computer vision have the potential to assist physios in video analysis. Action quality assessment (AQA) involves the quantification of how well actions are carried out. 2D and 3D human pose estimation (HPE) helps to identify the position of the skeleton subject. 3D shape estimation estimates the surface mesh of the subject. 3D force estimation can predict the ground reaction forces at each foot, and the torques at each joint. Our team including physiotherapists, biomechanical engineers and computer scientists will collect a dataset of videos from physiotherapists at a number of high-profile sports clubs. With labels provided by the physiotherapists, we can incorporate domain knowledge into the models.
The purpose of this proposal is to create a decentralised video dataset of physiotherapy strength and rehabilitation exercises by creating an app (building off the open source app of DataUnion) where a community of physiotherapists can record videos and add skeleton labels. The resulting dataset will be owned by the individuals that contribute to it. The contributions and ownership of the data will be handled in a data union. We are collaborating with DataUnion to make this happen.
- What problem is your project solving?
This project builds on the incentive and ownership infrastructure of co-owned datasets on Ocean Protocol, and the suite of data labelling tools of DataUnion to onboard a new community of physios and sports clubs to the Ocean ecosystem, as shown in the figure below. This solves problems related to a centralized Web 2 approach by (i) improving incentives for physios to provide domain knowledge to be used for building automated video assessment tools, (ii) providing ownership of a newly-created video dataset to the contributors, and (iii) crowdsourcing data labelling by physios using a new application that integrates with Ocean Protocol.
- Improving incentives by physios to input domain knowledge to the system
The physios have weak incentives to provide value - in the form of domain knowledge and time - by uploading videos and adding labels based on their own experience and domain knowledge. Typically, users of an app are incentivised to contribute data based on the results that are returned to them from a suite of models. This is a chicken-and-egg scenario in that we need the data to train accurate models, while we need accurate models to bring in data.
The contributors should be rewarded by ownership of the resulting dataset, more control over what it is used for and the ability to earn from models that are trained on the data. This project will create a proof-of-concept of a new incentive structure for crowdsourcing labels from physios. This will activate a new community into the data economy. The value proposition and market feasibility of this new approach will be further developed and evaluated, and a number of interviews will be performed with stakeholders.
- Providing ownership of data to physios and sports clubs
There is little infrastructure in place to provide data ownership to physiotherapists and sports clubs. Furthermore, the current state of ethics applications in companies and universities is not satisfactory to ensure control of individuals over data. The procedures in place are also slow.
This project will form a data union around this type of data and community. Some funding will be used to further develop a circular business model to scale the ecosystem safely and fairly, and for unlocking customers of associated data and models.
- Improving the uploading and labelling process with an app
Uploading and labelling videos is insecure and cumbersome. Tools for uploading and labelling videos will help to scale the process. Physiotherapists will log into the app and a digital wallet will be created for them. The users will upload videos and add annotations using a binary label to indicate whether the action was performed with good or bad quality. More sophisticated future models could have multi-class labels, indicating the attributes of the exercise that suggest good or bad quality and possibly give advice on how to improve the execution. The labeller will then verify that the data and annotations are correct and will be rewarded with shares of the dataset that they contributed to through tokens.
- What is the final product (e.g. App, URL, Medium, etc)?
A web-based application or mobile app for labelling videos of physiotherapy exercises.
- How does this project drive value to the Ocean ecosystem?
Firstly, the project will build an app to introduce a new community of physiotherapists to the ecosystem. These individuals may be attracted by the core values of privacy and ownership of data, and become stakeholders as a result. There were around 560.000 practising physiotherapists in the EU-27 in 2018. The app will onboard physios and create a digital wallet. We assume 0.01% of them (56 individuals) invests in OCEAN tokens as a result of this project with an average investment of 1000 $OCEAN. This results in a potential of a Total Value Locked (TVL) of 560.000 * 0.01% * 1000 $OCEAN = 56.000 $OCEAN demand.
Secondly, the project may introduce sports clubs to the ecosystem as new data publishers, who can be expected to have a higher stake than the average physiotherapist user. We assume that 1 sports club invests 30.000 $OCEAN, giving a TVL of 30.000 * 1 = 30.000 $OCEAN.
Thirdly, the project will create a unique new dataset related to health and sports that may attract a new type of investor in data tokens. We assume that an average high quality data pool has a TVL of 100.000 $OCEAN over one year. For a single initial dataset (combined across sports clubs), we get a TVL in data pools of 100.000 * 1 = 100.000 $OCEAN.
bang = 56.000 + 30.000 + 100.000 = 186.000 $OCEAN
buck = 18.000 $OCEAN
(% chance of success) = 80%
ROI = 186.000 $OCEAN / 18.000 $OCEAN * 0.8 = 8.2
This is above the expected ROI of 1.0.
Project Deliverables – Category
IF: Build / improve applications or integration to Ocean, then:
- App will be live, at: www.visiotherapy.ai or iOS AppStore + GooglePlay Store (url or app store)
- Software will be open-source with a permissive license based on existing work by DataUnion
IF: Outreach / community, then (one or more of):
- Blog posts will be published e.g. at medium.com, techireland.org, siliconrepublic.com - illustrating the community opportunities to increase activation
- A video production will be published at youtube.com, possibly including professional athletes at elite sports clubs in our community
- Outline possible circular business models, based on decentral set-up, and a respective roadmap
IF: Unleash data, then:
- Data will be made available on Ocean Market via compute-to-data
Project Deliverables - Roadmap
- Any prior work completed thus far?
Our team includes physiotherapists, biomechanical engineers and computer scientists and we have established relationships with a large community of physiotherapists and sports clubs. Furthermore, we have knowledge of 60 individuals and companies as potential customers and stakeholders. Previously, we have had conversations with 25 of these including:
- Leinster Rugby
- Irish Rugby Football Union
- Sydney Swans (5 times Australian Football League winners)
- Emovi (bio-medical device company for knee joint assessment)
- Irish Rowing
- Arsenal FC
- Manchester City FC
- Fulham FC
- Scottish FA
- Redbull Innovation Centre
- Badminton England
- …
Some clubs also indicated their interest in obtaining video analysis of training and matches from cameras in training pitches and stadiums. This touches on a new market of data and models for the sports analytics community that can be reached in future. From experience, other use cases include:
- Coaching and physio in markets for elite athletes of multiple sports
- Coaching and physio in markets for elite performers
- Coaching and rehab in markets for youth and development athletes
- Coaching and rehab in markets for amateur sports and performance
- Rehab in consumer market (e.g. tele-physio services)
- Coaching and rehab in market for at-home training
- Market for health and safety, workplace training
- Market for TV sports analysis
- Market for partners to leverage technology APIs or SDKs (e.g. for 3rd party applications)
- Teaching
- What is the project roadmap? That is: what are key milestones, and the target date for each milestone.
App Development
Jun 1 – 7: Design procedure for video labelling, in terms of number of labellers per video, verification etc.
Jun 7 – Jun 30: Work on feature for onboarding physios in app using digital wallets
Jul 1 – Jul 31: Work on feature for uploading videos
Aug 1– Aug 30: Work on features for annotation and validation of the annotation.
Business Model Development
Jun 1 – Jul 31: Develop a circular business model to unlock data and customers, and provide value to the community.
- Please include the milestone: publish an article/tutorial explaining your project as part of the grant (eg medium, etc).
Blog posts will be published e.g. at medium.com, techireland.org, siliconrepublic.com. These will be targeted especially at the physiotherapy community and potential customers such as sports clubs. Tutorials for uploading/labelling images will also be published.
- Please include the team’s future plans and intentions.
With a dataset of exercise quality created, we are going to train a set of models on this data and offer the use of the algorithm in the app to enable advanced assessment capabilities by users on 2D videos. This is a proof-of-concept for tele-physio services. Potential customers include less-experienced physiotherapists, strength and conditioning coaches, personal trainers or amateur clubs and athletes who want access to models that have built-in domain knowledge from physios at professional sports clubs. The model will also be sold via a marketplace on Ocean Protocol.
We will also expand the models on offer in the Ocean Protocol marketplace to include 2D and 3D human pose estimation (HPE). Human pose estimation is used to predict the joint positions of a human from images or video. These models will be useful tools to help physios assess exercise quality as both data labellers and customers - the algorithms will create a 3D skeleton on top of the 2D images.
To build a training dataset for 2D HPE, features will be added to the mobile app to annotate the skeleton of subjects in videos of exercises. 3D HPE models require motion capture datasets. We have close relationships with a number of universities, sports clubs and hospitals that have large MoCap datasets of exercises for physiotherapy applications. While generally happy to share, there are a number of hurdles that slow down the process. Firstly, ethical approval needs to be sought from a panel to share the data with each new group of researchers. Secondly, licensing arrangements need to be discussed every time with each new customer. These can take a long time. Also, there is no infrastructure for compute-to-data or federated learning in the biomechanics community. The data is thus duplicated on the servers of new research groups, which increases security risk. There is huge potential and upside to introduce Ocean Protocol to this ecosystem and our success story will pave the way to do so.
Core Team
Dr. Richard Blythman
- Role: machine learning engineer, biomechanical engineer, sports enthusiast
- Relevant Credentials (e.g.):
- Background/Experience:
- Video Intelligence Researcher at Huawei Technologies
- Research Fellow (Computer Science), Trinity College Dublin
- Machine Learning R&D Engineer at FotoNation, Xperi
Advisers
Dr. Mark Siebert
- Data publishing (10yrs)
- Business Development for Data Markets
- Owning and driving global executive engagements and partnerships, Data and Open Science
- Web3 experience: < 1 Year
- Positioning businesses in emerging markets or innovative fields with focus on data and AI-driven solutions
- https://www.linkedin.com/in/dsiebert/
Robin Lehmann
- Role: DataUnion/OceanDAO proposal onboarder
- Background/Experience:
- Head of Ocean Protocol Ambassador program
- CEO at DataUnion
- ML/Web3/DataUnion strategy at deltaDAO
- Machine learning (10yrs)
- Web3 (4yrs)
- Working on a bottom up approach to bring DataUnions to as many verticals as possible to learn and adopt the concept to their needs
- https://www.linkedin.com/in/robinlehmann/
