No results when using logstash

[swgroupw (Stuart Whitman)]

I am new to Elastic Stack and have been experimenting on a single CentOS 7 virtual machine. I have been following the installation and startup guides and have been pretty successful at getting things to work.

I am trying to get filebeat to work with logstash. If I configure the filebeat.yml file to use elasticsearch, results appear in the filebeat kibana dashboards. When I change filebeat.yml to use logstash, remove the /var/lib/filebeat/registry file and clear data with "curl -XDELETE 'http://localhost:9200/filebeat-*'" and restart filebeat the dashboards report no results found.

Using discover in kibana I see that the data is there, but most of the fields are not available when using logstash because they are empty. The fields are available and populated when using elasticsearch.

I have the syslog, logstash and auditd filebeat modules enabled.

Metricbeat works fine with logstash.

Any ideas what to try? I'll send whatever config files or log output needed. I didn't want to spam the list with unnecessary files.

[adrisr (Adrian Serrano)]

The problem here is that when you deleted the filebeat-* indices, you also deleted the index templates associated with them.

Beats will automatically set up those index templates for you when using Elasticsearch output, but cannot do so when using Logstash output. In this case you must load the index template manually before indexing any events.

Have a look at:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html

[swgroupw (Stuart Whitman)]

Thanks for the tip. Based on the link you provided I executed the following commands:

143 systemctl stop filebeat
144 curl -XDELETE 'http://localhost:9200/filebeat-*'
145 filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'
146 rm /var/lib/filebeat/registry
147 systemctl start filebeat

However, the filebeat dashboards still report no results found.