Could logstash distribute data to different nodes?

Elastic Stack Logstash
1

Hi, I want to distribute data into many nodes from logstash.
is it feasible?
because I want to reduce the risk. If one node crash others can share the loading.

thank you in advance !

2

Logstash instances are independent and don't cluster, but depending on what kind of data you're processing it can be more or less easy to add the kind of fault tolerance that you want. What kind of inputs do you have?

3

this is my logsatsh input:

input{
	tcp{
		
		port => 5510
		codec => json
		type =>"ntopng-*"
	}
	
}
	filter{
		if[type]=="ntopng-*"
		{
			if "" not in [IPV4_SRC_ADDR] and "" not in [IPV6_SRC_ADDR]
			{
				drop{}
			}
		}
	}
output{
elasticsearch {
				
                codec => "json" 
                hosts => ["localhost:9200"]
				user  =>elastic
				password =>SNOOPY255262

        }
	if[type]=="ntopng-*"
	{
		stdout{codec=> rubydebug}
	}
}

could I distribute the log or data to different hosts?
In order to reduce the risk that if one host crash, all system crash.

4

You can setup HA load balancer (Nginx or HAProxy) in front of your Elasticsearch nodes and use it for logstash output.

5

sorry, if my OS is windows 10, is it feasible?

6

Windows 10 doesn't look like recommended environment for production setup, but you can specify multiple hosts for elasticsearh output, i.e.:

hosts => ["hostA:9200","hostB:9200","hostC:9200"]
7 
hosts => ["hostA:9200","hostB:9200","hostC:9200"]

if I have "A","B","C" data, then it would distribute to hostA,hostB,hostC?
or only send to hostA,then if hostA crash, then it would send to B or C?

thank you in advance!

8

if I have "A","B","C" data, then it would distribute to hostA,hostB,hostC?
or only send to hostA,then if hostA crash, then it would send to B or C?

Quoting the documentation:

If given an array it will load balance requests across the hosts specified in the hosts parameter.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.