Logstash best practice

Elastic Stack Logstash
1

Hi,
we have a 5 nodes elasticsearch cluster running 5.2.0,
2 of the nodes are configured as client nodes (data=false, master=false),
3 nodes are elasticsearch cluster (data=true, master=true)

We want to install also logstash

What is the best practice for this topology?

Install the logstash at the 2 client nodes?

Install the logstash in new client server? - if so, what will happen if the node will fall? no logs will be written to elasticsearch?

Thanks,
Talia

2

What kind of inputs will your Logstash instance(s) have? What's the approximate event rate?

3

It will get a networking logs from switches from about 1000 machines
For about 300,000 events per half an hour

4

Okay, so a couple of hundred events per second. A single Logstash instance would have no problems coping with that. TCP or UDP? What load balancing options are available to you?

In general it doesn't matter so much where you install Logstash. Are you sure you need two client nodes? My hunch is that you'll be better off putting the money into more powerful data nodes.

5

TCP

So, if I will install the logstash at the 2 clients server it will work fine?

Another question, if I can,
How to I configure the elasticsearch to keep onle one mounth data?

Thanks a lot

6

So, if I will install the logstash at the 2 clients server it will work fine?

Sure, but it's not clear to me how you'll utilize both servers (hence my question about load balancing). As opposed to Elasticsearch, Logstash instance don't form clusters.

How to I configure the elasticsearch to keep onle one mounth data?

Look into Elastic Curator.

7

Sure, but it's not clear to me how you'll utilize both servers (hence my question about load balancing). As opposed to Elasticsearch, Logstash instance don't form clusters.

--> the load balancer is F5 with TCP

If I understand correctly F5 will do the balancing to the 2 clients servers and so I will have highly availability with the logstash also?
Also read this form:

and found that F5 is good for this configuration, Is it correct?

8

If I understand correctly F5 will do the balancing to the 2 clients servers and so I will have highly availability with the logstash also?

Yes.

9

Thanks

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.