New Ethereum talks, every Monday. The week's conference uploads by event, in your inbox.

Loading player…

Building the data layer for AI I Arpit Bindal, Rishikesh Kale and Abhay U

ETHGlobalThu, Oct 9, 2025, 12:00 AM

A three-part workshop on building the data layer for AI across Filecoin-based services. First, Synapse SDK: a full-stack Next.js demo that handles balances, USDFC approvals, on-chain payments, hot storage, and dataset creation—upload a file, fund allowance (approve → transfer → lock), and retrieve via CID. Second, Akave 0³ (S3-compatible) storage: AWS-style buckets, client-side encryption, immutable/versioned objects by default, Reed–Solomon erasure coding, and hackathon tracks (datasets, analytics, backup/recovery). Third, Lighthouse: permanent, encrypted storage with access control; upload via UI/JS SDK, share/revoke conditions; then mint “data coins” (e.g., POP) and reward users. Sample app: prove Amazon order history with Reclaim, verify server-side, store on Lighthouse, and mint POP tokens to users. 00:02 Intro & agenda: Synapse SDK, Akave 0³, Lighthouse 00:31 Prereqs: Filecoin Calibration FIL & USDFC faucets 00:54 Synapse FS-Upload demo app (Next.js full stack) 01:38 Synapse SDK overview & tutorial repo 02:10 Code tour: balances, payments, uploads, dataset hooks 03:08 Payment flow: approve USDFC → transfer → lock allowance 03:35 Running the app; what each tx does 03:59 Checking balances, rate allowance; deposit/increase allowance 04:23 Upload flow: pick file → init storage → connect provider → create dataset 04:41 Confirm on-chain tx; view dataset (CID) & download/retrieve 04:56 Store any data on Filecoin via Synapse SDK 05:21 Akave 0³: what it is (S3-compatible, cheaper, immutable) 05:54 Architecture: SDK → immutable ledger → distributed Akave nodes 06:57 Demo with AWS CLI (S3 endpoint set to 0³ RC2) 08:08 Create bucket; upload file; list objects 09:12 Versioning enabled by default; re-upload to create v2 10:14 Client-side encryption & Reed–Solomon erasure coding 11:28 Hackathon prizes & tracks (datasets, analytics, backup/recovery) 12:30 Starter guide & booth support 12:50 Lighthouse intro: permanent, encrypted, access-controlled storage 13:46 Log in & public upload via dashboard (CID/gateway) 14:12 Encrypted upload (sign, encrypt, then IPFS) 14:36 Create API key for programmatic use 14:58 Docs tour: upload (JS/API/CLI), uploadEncrypted, share/revoke, token conditions 15:58 Making data valuable: “data coins” as tickets for private data 16:18 Create a data coin on 1membb.io (POP example) 17:19 Chain, vesting, and locking assets; submit & programmatic script 18:05 Hackathon ask: users upload private data, get rewarded 18:31 Sample app (POP Rewards): prove Amazon orders with Reclaim 19:06 Flow: user proof → server verifies → upload to Lighthouse → mint POP 19:26 Wrap-up: tokenize datasets, attach value, earn from your project _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 🇮🇳 *ETHGlobal New Delhi* This workshop is specifically for ETHGlobal New Delhi, a 3-day hackathon held September 19-21 at the India International Convention & Expo Centre, bringing together the most skilled web3 developers, designers and product builders from all around the globe for a weekend-long adventure to advance the Ethereum ecosystem! Watch the full ETHGlobal New Delhi 2025 playlist here: https://www.youtube.com/playlist?list=PLXzKMXK2aHh7zCFW3hBqatZ18rQaYeEJL _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ✅ *Follow Filecoin* X: https://x.com/Filecoin ✅ *Follow ETHGlobal* X: https://x.com/ETHGlobal​ Warpcast: https://warpcast.com/ethglobal Website: https://ethglobal.com YouTube: https://www.youtube.com/c/ETHGlobal _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _Are you interested in Ethereum development and entrepreneurship?_ 👉 Sign up for the next ETHGlobal event: https://ethglobal.com/events 🎁 Get exclusive access and perks with ETHGlobal Plus! https://ethglobal.com/plus 📣 Want us to throw an event in your city? Tell us where! https://ethglobal.com/city

Transcript

[Music] Hello everyone, this is Rash and we are going to talk about what is Synapse SDK which would be followed by a short demonstration around Akawe which is our second track for Eid global New Delhi and the third track is for like lighthouse. So we would be also talking about how you would be able to integrate lighthouse and what's new around that. So let's start with having a look at the demo application of synapse SDK. All right. So let's just check out the FS upload tab which is a full stack application which integrates Synapse SDK with Nex.

js and uh you know like you can just quickly uh like integrate this inside your application in a similar manner. Uh but before that there are some prerequisite that you need to take care of. The first one is you need to get some testn net field tokens on fcon calibration test net. For that you can always reach out to this particular faucet and the next one is like you need to get some USDFC tokens as well. Now USDFC tokens are the stable coins on filecoin network.

So for now since we are doing it on like file calibration test net we would be getting some testn net USDFC tokens. All right. So you can just get that from the faucet too. Now the next step is like cing the repository. Uh you can also have a look at a detailed tutorial which we have which explains more or less almost everything that you might be needing to learn about what is synapse SDK the JS typescript SDK which is there.

Uh how the smart contact based market platform for collection of like services which are derived from the filecoin ecosystem. uh such as uh like which includes like the filecoin onchain payment services the hot storage uh and lots of like stuff around literable service we'll have a look at each one of them but like uh you can you know if you are following along with this video just start with cloning the repository installing the dependencies and just after that u you acquire all the test net tokens you can run the application using the command npm runde and you should be good to go before we actually head over towards the application which I have uh opened already in the other tab like I have the development uh you know set up ready uh I've already cloned uh uh the repository but before that there's some function that you need to you know like uh look into first one is like around like balances and storage mechanism which is followed by how the payments or like the functions that are responsible or like will help you to get the details around like USDFC uh which are followed by like file upload function alities and some data set related functions or hooks I would rather say. So these are all like the custom hooks which would enable you to you can just indicate this hook inside your front- end application and you would be good to go. So how does the payment work? It's like we need to first like get some rate allowance for which like we would be like uh approving some amount of USDFC.

The second transaction would be followed by transferring that and then last one would be locking up the allowance. So there will be like three-step process uh which we will be required uh like uh I guess like it's a good time now to actually head over towards application and see it in action. So yeah there's a short uh video demonstration as well which uh help which would help you to actually what's happening and like what things are required. All right. So yeah so pretty much like if you just see like whatever transactions which we are doing is something that we need to do on this application.

So at the moment my fill balance not 4 USDFC balance four good enough warm balance is awesome again rate allowance at the moment is like 10 GB which is more than sufficient and I have already uh made the uh you know uh got some allowance which uh I been doing in this particular like step. So you just need to deposit and increase the allowance as per your data set uh demand is. All right. So next step is uploading the file. So what you can do is just head over this and pick up any maybe like uh any like image for that matter.

Just hit the submit button. It will start to initialize the file upload and set up the storage service. It would then connect to a storage provider and once all the data uh pieces are ready to be added to a data set. It would start creating a data set for you. Uh as soon as you sign the transaction, it would wait for the transaction to be confirmed.

Once the transaction is confirmed, you would be able to see your data set in the view data set section. So all the data set which you have uploaded so far would be able to see you would be able to see it here. You can see the content identifier. You'll be able to download the image uh as well. And that's how like you can retrieve the data that you are storing onto this platform.

So you can store like any sort of data on filecoin onchain cloud which is powered by Synapse SDK. Hi everyone, I'm Abu Pades software engineer at AAVE. And in this workshop, we'll talk about what is a cave cloud. Uh what is a cave 03 and uh walk through a demo of a cave 03 and the tracks and prices offered by a cave at ETH global neutral. So first let's look at what is a cloud.

Basically, AA is a AA provides an S3 compatible API that is AAVE 03 which works well with already existing uh AWS S3 compatible SDKs and web 2 apps such as Snowflake and it provides up to 80% cost savings because of no eress cost and the files uploaded to AAI are imitable by default uh providing uh data integrity and verifiability on top of your data. and user uh user encryption user control data encryption and access AI agents can act agently on data. Okay. So now let's look at the architecture uh how a cloud that is a hosted a 03 service works. So let's say you are migrating a bucket from AWS S3 to AAVE 03.

How would that look? So first of all you'll use uh S3 I mean uh migration tool that is RC clone. So once data is injected using 03 into AAV the the data itself goes through a cave SDK and the metadata is stored in the immutable ledger that is AA's blockchain and the data itself is distributed among Akave nodes and stored in a decentralized manner. Okay. Okay.

So now let's look at uh demo. Okay. Let me move myself here. Okay. So for 03 demo, we'll be using Akavee uh sorry AWS CLI to demonstrate how compatible a 03 is with AWS S3.

And similarly you can use any SDK that is S3 compatible. And so let's look FYI, I've already configured my credentials using AWS configure. So I would so it would look like I'm just using S3 APIs, but in the back end I'm using uh yeah uh a 03. So let me show that to you. Okay.

So as you can see here in the default there is a region network and the endpoint UR is 03 RC2. That's what you guys will be using for uh hackathon. Okay. So, let's begin. AWS s3s.

So, this command lists all the bucket that I currently have. So, for example, I only have the test bucket. I'll quickly create a new bucket for this demo. A global demo. We just created a new bucket.

Let's list that bucket. It would be empty because we haven't uploaded any file yet. Okay. So, I have a test file which is uh 100 MB. Let's create 100 MB test file or 1 MB.

Okay. Now we'll copy this 1 MB tech uh test file to to our newly created bucket. So let's do that. As we can see the upload has been successful. Now we'll list the bucket again.

And as you can see here that is there is a test 1MP file. TXT that we just uploaded and the bytes are same. So there has been no modification in your data. Okay. So one more thing to uh point out is the versioning is enabled by default for a buckets.

So now let me showcase that to you. S3 API as you can see the version v1 versioning was enabled by default and we have currently version v1. Now if we edit the file and upload it again to the same bucket we'll see a version two of this file. Okay, let's uh let's create a let's randomize and create the same file with same name again. Now we'll copy over the same file once more to create a new version of same file.

Okay, let's take a look uh led out versions and as you can see the edited file that is the new file with the same name was uploaded as a version two uh uploaded with a version to version ID. So that's how you can version your uh files. So if you are in app or I mean app needs to modify the data somehow. So you can take this into consideration that the version uh versioning is enabled by default as compared to AWS S3. You don't need to do anything else.

You just need to upload the same file to the same bucket again with different data. Okay. So now let's take a look at what happened to our data once we uploaded it. So client side encryption if you want to encrypt your data you can enable the encryption to uh to the CLI or the SDK whatever you using then the data gets encrypted and the encrypted data blocks are eraser coded using read solommen uh erasure coding algorithm and the data blocks and parity blocks are created. So once this uh when all the data blocks are ready it's uploaded to a cover network.

So up until here everything's happening client side the data is getting it recorded client side and the data is to a network. Then when you are trying to retrieve that file this blocks are collected again and using readment recovery if the data is lost it will be repaired and the original file will be uh served to you. Okay. So let's talk about price structure uh during this hackathon. So we have three prizes that is 12 uh $1,250, $750 and $500 uh respectively and the few trades.

So this time we we are not giving an open challenge statement but a closed one. uh to be eligible for submission you need to you need to lie among one of these tracks. So you can choose a IML data sets and checkpoints cross log analytics that is using snowflake ice uh apache iceberg and and the pin ingest or immutable data backup or uh data recovery. So if you don't know where to start, we have provided a starter guide uh in the ET global dashboard. You can go there and take a look at it or find us at our booth, protocol labs booth, and we'll be able to help you out.

So that's it about a thank you for watching. I'm Akash, a software engineer at Lighthouse and in this workshop, I'll take you through the lightest bounty for ETH global New Delhi. Quick intro. I built developer tooling at Lighthouse. So you can store data, keep it private and access control them directly from your app.

So I'll show you an end to- end flow on how you can upload data, control access and what do we expect to see at the hackathon. Plus I'll also show you a sample app towards the end. So first let's talk about Lighthouse. Lighthouse is the permanent decentralized solution on Filecoins with built-in encryption and access control. You can upload a file publicly or encrypted fetch them via CD from IPFS gateway and you can also control them with access conditions with onchain access conditions like a wallet which holds a certain token or a address which is on the allow list so you can access the file.

Now you can do everything from the UI or programmatically by the JS and the Python SDK. Now let's head over to thei file step and see how we can upload a file. Okay, now let's log into Lighthouse. So either you can login via email or Google or GitHub account or either you can choose one of the provided wallets. Let's choose MetaMask for now.

I'll click confirm and I'm logged into the app. Now let's try to upload an image on the dashboard. I'll click on upload new. Upload a file and I'll choose the image. That's it.

You'll be able to see the C here and you can click and open the image. Now if you want to upload an encrypted image, you can just turn on the encryption from here and then upload a new file again. I click on this. So it will ask me to confirm the signature. I click confirm.

So what happens here is that the file is going to be encrypted first and then it will be uploaded to IPFS. The file stored is encrypted and required decryption right to access. Now, yep, the file is uploaded and for the programmatic use, let's head over to the API key and create an API key for now. Let's give it a name of test one. Are you sure you want to generate?

Yes. So, this API key isn't stored on Lighthouse. So, you won't be able to recover it. So, just either copy it to the clipboard or download the key. So, now that as the API key is ready, I'll show you how you can locate the docs.

Okay. Now let's head over to the docs. Now here you have four essential tabs to look at. First one is uploading a file. So in this you have three methods to upload a file using the JS SDK or the API or the CLI.

The JSDK has two methods to upload. First one is in the NodeJS or in the browser itself. Then you have a text upload using the lighthouse.load text. You can use it to upload any text on the lighthouse.

The next one is the lighthouse.upload upload encrypted which is used to upload any encrypted file on the lighthouse itself. It again has two methods. First one is a nodejs and another is the browser and the last one are the encryption features. There are multiple encryption features that you can use in the lighthouse itself.

First one is the share file which can be used to share any file and access to any file to a wallet address. You can also revoke the access using revoke access or set any token conditions, any token getting conditions to a file and use the check access condition to view those conditions. Okay, cool. Now we have got the basics covered. But how do we make this data worth anything?

How do the contributors get paid for their data? Here's where the data coin shows up. Think of them as a ticket to the private data. So let's head over to 1membb.io and let's create a data coin.

Okay. So this is the website. Click on the create data coin. Now here it will ask you to enter the information of the coin. Let's enter the information name.

Let's give it pop rewards. Pop stands for proof of purchase. I'll give it a symbol of pop. Uh let's upload an image. This is the logo there.

Fine. This is the banner image. Okay. Uh for the website, let's give it https rewards.io.

Let's click on next. Here you can enter the social information of the token. I'm skipping it for now. And here you can enter the token allocation information. So we can select the chain from here.

Let's select past sapoleia for now or maybe eat sapolia for now. Here you can set the vesting period. I'm keeping it to 6 months. Then you can keep the other default information as it is. And for the locking assets, you can select either of these.

I'm selecting USDC and the locking amount to be $100. And I click on submit data. So now the data has been created successfully. If you prefer a programmatic approach, we also have a script for you to create the data coin. This is the script and the link to this GitHub repo will be there in the description.

Now that the data coin exist, here's what we want you to make for the LAR bounty. Build a small project where real users can upload private data and earn something useful right away. We want you to take the user's private data, verify it, maybe using GKLS, and then upload it on the lighthouse. If you think that the data is sensitive and useful, encrypt it with a access condition and create the decryption with an allow list. Let me show you a sample app that we have created for you to understand the concept.

So this is the pop rewards. It's a proof of purchase app for your Amazon order history. So basically this app lets you prove your Amazon orders and gives you reward on the basis of that. So I'll start by clicking on start earning. I'll choose MetaMask.

Right. So here I am logging in as a user to see the app. As soon as I'm logged in, I will be redirected to the dashboard. So this is the dashboard and here the here you can see all the proofs that the user has submitted in the past. Now if I click on the add data, it will take me to the reclaim page.

So this is the reclaim QR code that the user can scan and prove their Amazon history from from with the help of reclaim. So as soon as the user proves his order history, the reclaim will give us the proof on the front end and we'll verify this proof on the back end and after the proof has been verified we will upload the user's data which is relevant to us that is the order history in this case to the lighthouse and after uploading the data to the lighthouse we will mint some pop tokens that we have created in the last step to the user's wallet. That's it. That's the loop. So turn your data set into a token.

Attach your attach a real value to it and let people who need it pay for the access. Congrats. Now you have now you can earn from your hackathon project. That's it about the lighthouse. Thank you for watching.

[Music]

Automatic transcript — names and jargon may be misspelled.