# Seeking a developer to help extend Elasticsearch to connect with Web3 API **URL:** \ **Category:** Elasticsearch\ **Created:** [June 15, 2023, 3:06pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114 "2023-06-15T15:06:02Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Jules\_Lai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jules_lai/32/122308_2.png) [@Jules\_Lai](https://discuss.elastic.co/u/Jules_Lai)\ **Post date:** [June 15, 2023, 3:06pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/1 "2023-06-15T15:06:02Z")
Hi, I am looking for a developer who is able to help integrate Web3 into Elasticsearch. I am one of the developers working on the Web3 interface for SIA decentralised storage. I will be mainly using Elasticsearch with AI LLM models. I am looking to store the Elasticsearch index on the decentralised storage. The plan is that this will be open-source so that anyone can use. Do you have relevant experience and can you help me? Best, Jules Lai (CTO of Fabstir) ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 15, 2023, 3:22pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/2 "2023-06-15T15:22:52Z")
It looks like there's a S3-compatible interface for Sia already, so it should already work as a backend for [snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html) and [searchable snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/current/searchable-snapshots.html). Apart from that, Elasticsearch doesn't really work well with high-write-latency storage today. That's one of the key areas of innovation for the [stateless](https://www.elastic.co/blog/stateless-your-new-state-of-find-with-elasticsearch) architecture on which we're currently working. Again, AWS S3 is one of the possible backends. ---
**Author:** ![Jules\_Lai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jules_lai/32/122308_2.png) [@Jules\_Lai](https://discuss.elastic.co/u/Jules_Lai)\ **Post date:** [June 17, 2023, 10:32pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/3 "2023-06-17T22:32:52Z")
Sorry for my delay to reply. Thanks for your answer. The S3 interface is for Amazon's storage, not Web3. I'm thinking of Web3 ways of keeping the backups censorship resistant and for it to be private and under ownership of users rather than a centralised platform or authority. How easy would it be to code it so that snapshots get stored directly to decentralised storage. Is there an API that I only need to implement for it to integrate into Elasticsearch? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 18, 2023, 2:18am UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/4 "2023-06-18T02:18:50Z")
You can look at this plugin to get some inspiration: > **[elasticsearch/plugins/repository-hdfs at main ยท elastic/elasticsearch](https://github.com/elastic/elasticsearch/tree/main/plugins/repository-hdfs)** > > main/plugins/repository-hdfs ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [June 18, 2023, 6:17am UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/5 "2023-06-18T06:17:18Z")
> [@Jules\_Lai](#): > > Thanks for your answer. The S3 interface is for Amazon's storage, not Web3. Sorry if I'm misunderstanding something, but [https://filebase.com/](https://filebase.com/) claims to be a S3-compatible interface to Web3 decentralised storage. It sounds like exactly what you want. ---
**Author:** ![Jules\_Lai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jules_lai/32/122308_2.png) [@Jules\_Lai](https://discuss.elastic.co/u/Jules_Lai)\ **Post date:** [June 19, 2023, 3:30pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/6 "2023-06-19T15:30:15Z")
Thank you for the code reference. I'm getting a picture for what needs to be done. I will put together a plan/proposal. What I want to do is put out an open-source framework that will allow AI models that can use Elasticsearch to store its snapshots to decentralised storage. From there to open up Web3 use cases. ---
**Author:** ![Jules\_Lai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jules_lai/32/122308_2.png) [@Jules\_Lai](https://discuss.elastic.co/u/Jules_Lai)\ **Post date:** [June 19, 2023, 4:42pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/7 "2023-06-19T16:42:34Z")
It could be but S5 does use a CDN for better latency etc., but after talking to the developer, the s3 compatible interface has not been built yet. My plan is to use the s5 web3 client to store the snapshot serialised data to s5 SIA more directly and have better performance benefits. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 17, 2023, 5:07pm UTC](https://discuss.elastic.co/t/seeking-a-developer-to-help-extend-elasticsearch-to-connect-with-web3-api/336114/9 "2023-07-17T17:07:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.