Project Coral | Persistent Data Tokens: Decentralized Data Archival & Access with Filecoin + IPFS Integration | Round 16

Project Name

Project Coral

Project Description

I am requesting a continuation for our grant to execute our timeline for bootstrapping an existing community of neuroscientists to begin working with the Ocean Protocol and unleash neuroimaging data.

Final Product

not set

Proposal One Liner

This grant will fund development work to improve upon Ocean Protocol by automating decentralized file storage and archival for published datatoken contents.

Proposal Description

Summary

For round 16, OpSci is adding native support to Ocean Provider for Filecoin archival and publishing data to IPFS. Ocean Protocol can not currently guarantee that a URL gated by a data token will always be resolvable. Google drive, AWS buckets, and personal server links can sometimes become unavailable. OpSci is solving this problem by archiving data on Filecoin, making it persistently available for a set timespan, and pinning to IPFS for peer-to-peer sharing. Persistent data tokens are data tokens that can guarantee the availability of their content over a period of time

A Pull Request will be made on the Ocean Market front end GitHub - oceanprotocol/market: :mermaid: THE Data Market and Ocean Data Provider back-end https://github.com/oceanprotocol/provider. We commit to working with Ocean core developers to merging the PR, following software quality best practices.

Software Overview:

Users interact with the Data Locker UI embedded within the Ocean Market. A simple drag and drop results in an HTTP request to an external server running Estuary, a hybrid IPFS gateway + Filecoin Auction node.

Estuary has built in API access for creating collections, adding to collections, managing PINs, preparing data for a Filecoin storage auction, and pinning to IPFS.

An internal database tracks uploaded files as they are prepared (converted into a serialized DAG, or “CAR” file) for auction and a deal is made to be stored across a minimum of 6 miners. Architecture details appear below.

Architecture for Storage

Note that the current API requires a key to use the public service offered by ARG, the maintainers of Estuary. Users interested in this service can apply for an API key through Protocol Labs, or run their own estuary node. This use case may be appropriate for independent data providers that are running their own software and seek to automatically archive or serve their own data via IPFS or charge users access to archival.

The Ocean Market Web UI will be updated to support the new functionality. We plan to update the profile page with a Data Locker stats section dedicated specifically to storage deals and data uploads. This first variation will be a simple implementation with just key overview information.

Profile New

We will also add a new tab to the Ocean Market to allow users drag and drop functionality. Users may also choose to archive a file to Filecoin directly using a CID of an existing IPFS pinned file.

Upload

Grant Deliverables

[ ] Persistent data tokens: Archive data token content on Filecoin and pin on IPFS

[ ] User account dashboard for uploaded data

[ ] Propose metadata spec to include archival and pinning metadata

Value Add Criteria

The addition of the Filecoin/IPFS archival feature will provide new revenue avenues for others that utilize the tech stack and choose to run their own data provider node. Examples include charging for archival, pinning, or retrieval.

Funding Requested

20000

Wallet Address

0xf023D9b047243B911e132E4B5877b5f09B8B66B9