How to index s3 files and transfer that index to my ES server

Elastic Stack Elasticsearch
1

Hi,

I have been indexing files from different servers using FSCrawler and storing them in my Elastic search server.
But now I have files in s3 bucket. Could anyone please guide me how to index files that are in S3 and store them to my ES server?
From the previous discussions, I got to know I need to use log stash-s3-input plugin, but I am not able to find the procedure to do that.
Another novice question, Can I use my ES server to store the s3 index or is it necessary to have a EC2 instance to index s3 files?
Is it possible to store all my indexes in normal ES cluster instead of using Amazon ElasticSearch??

Please guide me or post a link that explains s3 indexing for beginners. TIA!!

-Lisa

2

Anyone???

3

May be you'd like this not implemented feature?

I don't know S3 enough but may be you can mount the s3 repo as a mounted drive and index the drive with FSCrawler from the ec2 machine?

4

Thank you so much!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.