How to use FSCrawler with Azure Blob Storage?

Elastic Stack Elasticsearch
1

I want to index the documents stored in Azure Blob Storage into Elasticsearch.
Is it possible to use FSCrawler in this case by providing the blob container URL?

2

Hi @AbderrahimAl
Welcome to the community! Perhaps look at this

1 Like
3

Hi @stephenb,
Thank you for your response!

I am not familiar with Filebeat, but I think the azure blob storage input support only JSON and NDJSON format, please correct me if I am wrong.

In our case, we are interested more in other formats like pdf, docx, csv ...

Maybe the idea I will try next, is to use Attachment processor in elasticsearch and create a serverless function that takes blob storage as a trigger, encode the file in base64 and send it to elasticsearch ingest pipeline.

1 Like
4

Is it possible to use FSCrawler in this case by providing the blob container URL?

Not yet. But I'd love to support this at some point.

3 Likes
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.