Standalone Logstash output to external Elastic Search server

Elastic Stack Elasticsearch
1

HI, have been struggling to get the output of one logstash servers into an elastics search server. I have also opened the discussion on logstash however no solution was found there for the challenge until now, as such I would like to find out if this could possibly be a ES challenge. When doing a tcpdump on the ES installation I can see the server is connecting, however when looking at the indexes of ES there are none created for the external logstash server. The output of the logstash server loos as follows:

output { elasticsearch { hosts => [ "192.168.163.21:9200" ] sniffing => true flush_size => 20 idle_flush_time => 15 index => "IDNS-" } stdout { codec => rubydebug } file { path => "/var/log/syslog/ES/IBIND-%{+YYYY-MM-dd}.log" codec => "rubydebug" } }

I can telnet to the port from the logstash server to the ES server:
[root@localhost ~]# telnet 192.168.163.21 9200
Trying 192.168.163.21...
Connected to 192.168.163.21.
Escape character is '^]'.

The index is however not available on the ES server:
[root@syslog ~]# curl '192.168.163.21:9200/_cat/indices?v'
health status index pri rep docs.count docs.deleted store.size pri.store.size
yellow open logstash-2016.02.28 5 1 1581621 0 1.1gb 1.1gb
yellow open logstash-2016.02.27 5 1 1823938 0 1.2gb 1.2gb
yellow open logstash-2016.02.26 5 1 1856485 0 1.3gb 1.3gb
yellow open logstash-2016.02.25 5 1 2068778 0 1.4gb 1.4gb
yellow open logstash-2016.02.19 5 1 1406589 0 1gb 1gb
yellow open logstash-2016.02.18 5 1 399705 0 300.4mb 300.4mb
yellow open logstash-2016.02.29 5 1 1211654 0 927.8mb 927.8mb
yellow open .kibana 1 1 11 1 37.9kb 37.9kb
yellow open logstash-2016.02.20 5 1 1326447 0 955.9mb 955.9mb
yellow open logstash-2016.02.24 5 1 1866926 0 1.2gb 1.2gb
yellow open logstash-2016.02.23 5 1 1737767 0 1.2gb 1.2gb
yellow open logstash-2016.02.22 5 1 1506259 0 1gb 1gb
yellow open logstash-2016.02.21 5 1 1366443 0 933.5mb 933.5mb
[root@syslog ~]#

The ES configuration looks as follows for file elasticsearch.yml:
`[root@syslog backup]# cat /etc/elasticsearch/elasticsearch.yml

http.port: 9200

[root@syslog backup]#
`

I hope someone would be able to assist.

2

Hi,

For your index pattern can you use:

index => "IDNS-%{+YYYY.MM.dd}"

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index

Also you can run logstash with --debug and it will give you more details as to whats happening.