Docker Elasticsearch 6.4.0 and Logstash


(Das Wild) #1

Hey there,
I'm very new to Docker and Elasticsearch so I hope you can help me.

I created a ES Cluster with 5 nodes (2x dedicated master nodes, 2 data/ingest nodes and one coordinate node). My containers look like:
es_master01 0.0.0.0:9201->9200/tcp 9300/tcp
es_master02 0.0.0.0:9202->9200/tcp 9300/tcp
es_data01 0.0.0.0:9203->9200/tcp 9300/tcp
es_data02 0.0.0.0:9204->9200/tcp 9300/tcp
es_client01 0.0.0.0:9200->9200/tcp 9300/tcp

When I do a curl http://localhost:9200/_cat/health I get following output:
1536836917 11:08:37 elastic_cluster green 5 2 0 0 0 0 0 0 - 100.0%

And on curl http://localhost:9200/_cat/nodes this one:
172.18.0.3 31 28 2 0.21 0.21 0.71 m - es_master02
172.18.0.6 47 28 2 0.21 0.21 0.71 - - es_client01
172.18.0.2 29 28 2 0.21 0.21 0.71 m * es_master01
172.18.0.5 24 28 2 0.21 0.21 0.71 di - es_data02
172.18.0.4 32 28 2 0.21 0.21 0.71 di - es_data01

After that I setup a container with logstash and opened a new pipeline with a TCP input and a output to Elasticsearch wich looks like:
input {
tcp {
port => 1234
}
}
output {
elasticsearch {
hosts => ["http://es_data01:9200","http://es_data02:9200"]
index => "test_tcp-%{+YYYY.MM.dd}"
}
}

But the expected index is not created. Pinging between the nodes is no problem. Do you have any idea where to search for a solution?

Thanks a lot

Stephan

EDIT:
I created another output:
file {
path => "/tmp/test-%{+YYYY.MM.dd}"
}

and this works fine. So I think it's a problem of networking or elasticsearch.


(Das Wild) #2

Problem is solved!
Logstash will not send to an elasticsearch node with a "_" in the name...


(Toby McLaughlin) #3

Glad you found it!

It's worth mentioning that underscores are forbidden in hostnames in general (RFC 952).


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.