Build & Integrate
We would like to add a metadata feature where we inspect the datasets and detect the sensitive data by leveraging AI (Convolutional Neural Network (CNN)). The corresponding columns in the CSV file format that was detected as sensitive data will be ignored when we run the Compute Job by reading the results of the Sensitive Data Inspector Module in JSON when we configure the dataset path for the given algorithm. In this way, we will provide complete sensitive data security and also the “training data” concept. We will also assess the quality of the data by scanning through the data points to make sure that the main dimensions of data quality exist based on the relevant KPI that was applied in the AI Model.
Grant Deliverables 1: Sensitive Text Data Inspector Function powered by AI
- Output a result with a ratio indicating the columns that possibly contain sensitive data
Grant Deliverables 2: Qualitative Text Data Inspector Function powered by AI
- Output a result with the ratio indicating the qualitativeness of the dataset that is based on the KPIs applied
Grant Deliverables 1: Sensitive Text Data Inspector Function powered by AI (Convolutional Neural Network (CNN))
- Sensitive Data Inspector Function powered by AI - Development completed & System test started - Apr 15, 2022
- System test by developer completed & functions published on SwaggerHub - May 13, 2022
- Test cases and sample datasets will be provided for Acceptance Test - on May 20, 2022
- Acceptance test on Swagger - May 27, 2022
- Publishing on social media that we release the pre-beta version - on June 30, 2022
- Pre-Beta testers will be informed - on July 1, 2022
Grant Deliverables 2: Qualitative Text Data Inspector Function powered by AI (Convolutional Neural Network (CNN))
- Qualitative Data Inspector Function powered by AI - Development completed & System test started - July 30, 2022
- System test by developer completed & functions published on SwaggerHub - Aug 13, 2022
- Test cases and sample datasets will be provided for Acceptance Test - on Aug 20, 2022
- Acceptance test on Swagger - Aug 27, 2022
- Publishing on social media that we release the MVP beta version - on Aug 30, 2022
- MVP Beta testers will be informed - on Sep 1, 2022
- Inspector Module Functions in Python
- GitHub will be used for Code & Version Control
- Inspector decision-making intelligence by Convolutional Neural Network (CNN)
- Visual Studio Code will be used as IDE
- Inspector results will be generated in JSON
- Functions will be published on SwaggerHub
We will maintain and develop further and fix bugs/errors since this module will be part of our Datatera solution.
- Dataset format will be in CSV only from the beginning and we can definitely support more formats e.g. XML, Xls, etc., and even medical images.
- We will probably add more KPI and metrics to be able to better detect the sensitive and qualitative datasets.
- We will add possible extensions to this work to be able to provide more relevant AI insights on the metadata feature.
Datatera is a global marketplace to connect HealthData Providers with HealthTech companies by making larger samples of the high-quality real-world datasets available.
HealthTech AI companies are facing challenges to get access to qualitative healthcare datasets while they are building AI models which result in bias and other errors and that take a lot of time and money to maintain and manage. Datatera will provide a global data computing marketplace where Data Scientists will have the opportunity to train their AI models on high-qualitative and diverse training datasets while preserving privacy.
Usage of Ocean and Viability - We have an intention to enrich the metadata feature with very valuable AI insights to be able to help Data Consumers to choose the right dataset for their needs to consume.
We believe we can improve and develop the C2D concept with a richer metadata feature to be able to provide full awareness of data sensitivity and data quality of the datasets we provide in our platform.
It is equally important to ensure that all datasets that are available on our platform were already inspected and they contain certain value to Data Consumers when they choose to train their AI models.
Role: Developer, CTO, Lead Developer, Architect
Founder at Datatera
10+ years of professional experience as a senior system developer
Role: Senior Pharmacist & Data Analyst
Data Analyst at Datatera
Role: Senior Data Analyst
Senior Data Analyst at Datatera
Data Advisor at Datatera
Senior BI Data Scientist
Medical Data Advisor at Datatera
Medical Doctor in radiology and Ph.D. in Artificial intelligence for prostate cancer detection.
Minimum Funding Requested