Best practice on data ingestion methods

Elastic Stack Elasticsearch
1

I currently have a self managed elasticsearch cluster that up until now I have just been feeding with data via HTTP POSTs (to port 9200) to specific indexes from various servers.

I'm now in the process of setting up beats and Logstash which will provide new/better pipelines.

I just wondered what the general best practices are around just feeding data into elasticsearch via direct HTTP POSTs? Is this something that shouldn't be used in production, or is it something that has legitimate use cases? I'm considering whether to keep it as an option or move away from it completely.

When I research ingest methods of course people talk about beats, Logstash and API clients but I don't see much discussion about direct HTTP POSTs.

2

Welcome to our community! :smiley:

I would probably look at using more ingest pipelines in Elasticsearch over Logstash. If you take that route you can then use coordinating nodes to reduce any direct pressure on data nodes in your cluster.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.