Unity to Ocean Uploader | Unity Implementation - Ocean Marketplace | Round 17

OceanDAO Round 17
#1

Project Name

Unity to Ocean Uploader

Project Category

Build & Integrate

Proposal Earmark

New Outreach

Proposal Description

A Unity Editor tool, that let’s you share a scene, models and scripts to the Ocean Marketplace in order to share it with other people easily.

Grant Deliverables

Editor tool (script) on unity’s nav bar, that let’s you choose a scene to upload it to the Ocean Marketplace.

Project Description

A Unity Editor tool, that let’s you share a scene, models and scripts to the Ocean Marketplace in order to share it with other people easily.

Final Product

Editor tool (script) on unity’s nav bar, that let’s you choose a scene to upload it to the Ocean Marketplace.

This project is a part of a bigger project, AI Drone Driver (name to be updated)

Description:

Right now most automatic drones with AI that are bound to them or to a server, which handles directly the input and creates the output, which can be sometimes slow and have issues and are target specific (only same drone etc)…

AIs can work as actual drivers, they will be connecting through easy API connections with drones (e.x dji controller api), will collect input data from it and will generate the output remotely, this way they can coordinate multiple drones, at once, while the drones themselves will be “mindless”. This architecture would make drone swarms act better together.

Features:

  • Simple to use Android, iOS & Desktop (Windows, Linux, MAC) app to connect to remote drones (front-end - client)
  • Middle Server that handles the world building (unity or nodejs), a headless app that connects with Thoth (AI) and with the Client, will generate the world around the drones and get world-wise information to the AI
  • Everything will communicate together using TCP sockets, for ease of use and faster movement of data.
  • The AI will be able to generate commands based on the input from the Drones and the MiddleServer, will be fetching objects around each drone, real-image visuals with objects detected (will be spawned in the World Server too)

Architecture:

The Project will consist of 3 Apps:

  • The Client (will be accessible to users and with this will be able to update the data and drones, also will have the connections with the drones, will be able to run on Windows, Linux, Mac, iOS or Android…
  • The MiddleServer/World Server, will be built with Unity or NodeJS, will create a 3D world with the world and the drones, will be a headless server, which will be able to run on Linux, Mac or Windows (headfull too).
  • The backend Server, which will be Thoth, it will handle the AI directly, will be connected to the World Server (TCP) and to the Client (Web Requests), will be doing all the heavy lifting and generate the output for each command. Thoth also includes a web editor (node graph) to easily create AIs.

Tech Stack:

Front End Application:

  • The client will be the API connector with the drones, it will be able to generate multiple connections at once.
  • Will be able to get information from the drones (camera, altitude etc)
  • Will send the generated commands (move towards, strafe etc) to the drone(s)
  • Save/Load Drone Formations, default speeds, max altitude etc

Middle Server - World Server:

  • Backend will generate a 3D world around the drone at a smaller scale with information visible to the camera, which will work like a small rendering game camera
  • The world will include all the drones that are connected to it
  • Each AI in the world will have a unique id, based on the client id for it

Back End Application - Thoth:

  • Thoth will include all the models and AI data that will be needed to generate the commands
  • Includes a Web Editor (Node Graph, like UE4) that lets the user create easily graphs for the AI behavior

Models Needed:

  • Computer Vision: Visual Object Detector - Python algorithm using ImageAI/Tensorflow to detect objects in images, includes scripts for various clients (screenshots, images, videos, game windows)
  • Decision Making: Will use NLP for decision making, probably OpenAI (GPT3) or GPT-J (which is open source and self-hosted). This will create a Word-Based World, which the AI will use and will send word-commands for the movement
  • Maybe More?

Drone Prototype:

  • There will be a prototype drone with more sensors needed for the best use of the AIs (although it will work normally with other types of drones too)
  • It will include all the basic sensors, but also a Thermal Camera and a Night Vision camera, for better results during night.

1200×1600 339 KB

Web3 Integration - Blockchain:

  • Save/Load and Mint Plans for Drones (Formations, good speeds etc)
  • Static Website in Blockchain- Manager for the World Server/Drone Manager
  • Login System through Blockchain
  • Pub-Sub system using Blockchain Canisters ?
  • Implement Blockchain to unity Client for login and save/load data

Also, we’ve won the Grant from Algovera AI for the Visual Object Detector

And used OceanMarketplace for the main script

Value Add Criteria

It will enable easy integration for the Ocean Marketplace in unity.

Core Team

Alexandros Titonis

Role: AI Engineer & Full Stack Developer

Github: alextitonis · GitHub

LinkedIn: https://www.linkedin.com/in/alexandros-titonis-0ba061176/

Experience:

I’ve been working lately on Thoth a Node Web Editor for AI, i’ve worked on webaverse and xr-engine.

In previous years i focused working on Multiplayer games in Unity.

Funding Requested
3000

Minimum Funding Requested
500

Wallet Address
0xd5bDc028aaEAb8D8fa1C5Bb577b6C4C3402559F6

3 Likes
Algovera | A Decentralized Hub for Data Scientists in Web3 | Round 17
#2

Thank you for applying for R17!

Your proposal has been registered into the system and everything looks great!

Thank you for submitting your proposal for R-N!

I am a Project-Guiding Member and have assigned myself to help you.

I have reviewed your proposal and would like to thank you for your participation inside of the Ocean Ecosystem!

Your project looks promising and I believe it’s aligned with our evaluation criteria of generating positive value towards the Ocean Ecosystem and the W3SL.

I would also recommend one (or all) of the following to increase support:

  1. Saying hi to the community in #ocean-dao and sharing your proposal.
  2. Saying hi to members of the #project-guiding WG and sharing your proposal.
  3. Meet with the Guides assigned to you by the #project-guiding WG.
  4. Attend a Town Hall or Project-Guiding WG meeting to talk about your project and proposal.

I am in support of your project and proposal. I look forward to continuing providing support and feedback to your project, and you can expect to receive a positive vote from me during the upcoming voting period.

All the best!

  • Timothy C | OCEAN PGWG
1 Like
#3

Project submitted deliverables:

To run you need Nodejs LTS and Unity (tested on version: 2021.3.1f, but should work with other versions too).

First run

  • cd Assets/Scripts/Editor/ocean

  • npm install

#4

Admin:

Hi Alex, great work and thank you for submitting your deliverables. I have reviewed them, and although I’m familiar with Unity, many others are not. I am going to reject your submission for now as discussed, and ask you to provide a more digestible/non-technical way of understanding your progress, so we can accept your completion. Thank you!

#5

Information:

Unity To Ocean Uploader is a tool that can upload automatically a unity scene to Ocean Marketplace through Dropbox. It packs automatically all the dependencies needed for the scene and uploads it with the click of a button. Small Video: Unity to ocean uploader - YouTube
How to run it:

In order to run it you only need to select the scene, set the dropbox api key and the nodejs local part.

cd Assets/Scripts/Editor/Ocean
npm install
Open unity, select a scene and setup the api key and path