Logstash does not output to ES

Elastic Stack Logstash
1

I am new to logstash and have worked to configure a basic pipeline. However, this does not seems to output anything into Elasticsearch.
I can see it send output to stdout though. I am hoping that someone can look over my settings and perhaps identify where I went wrong.

Contents of Pipelines.yml

  • pipeline.id: main
    path.config: "/etc/logstash/conf.d/beats.conf"

Contents of beats.conf

input {
beats {
port => 5044
}
}

output {
elasticsearch {
hosts => ["http://elasticserver01@domain.com:9200", "http://elasticserver02@domain.com:9200", "http://elasticserver03@domain.com:9200"]
index => "%{[@metadata][beat]}-%{[@metadata][version]}"
}
stdout{}
}

Contents of logstash.yml

node.name: server01
path.data: /var/lib/logstash
pipeline.id: main
pipeline.batch.size: 125
pipeline.batch.delay: 50
path.config: /etc/logstash/conf.d/beats.conf
config.test_and_exit: true
dead_letter_queue.enable: true
dead_letter_queue.max_bytes: 1024mb
path.dead_letter_queue: /var/lib/logstash/dead_letter_queue
log.level: info
path.logs: /var/log/logstash
logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = info

Logstash-plain.log has not entries since November where I was working on getting Logstash installed and set up.

logstash-slowlog-plain.log is empty

Please let me know if I need to post more info.

2

logstash just verifies that the config can be compiled when you set this. It does not run the pipeline.

3

I am pretty sure the pipeline is running. If I remove the stdout config from the pipeline config it won't write anything. When I comment it back in I can see it in the logs. So this makes me thing there is an issue with Elasticsearch section of the outputs, but I can't think of what I am missing.

4

So I changed to the index in the conf file to "testindex" then searched for that in Kibana,

GET _cat/count/testindex

1576687443 16:44:03 1131

So clearly I am getting documents into the index, when I check the Logs tab in Kibana though I don't really see anything. Anyone know what I am missing here?
I am using default mapping if that matters in regards to which fields are searchable.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.