Changing winlogbeat from elasticsearch to logstash

Hi,
I started off a cluster with winlogbeat going directly to elasticsearch and using the pre-built dashboards. All that worked well out of the box. Now I'd like to send my winlogbeat data through logstash so I can do some email alerting out of logstash on the winlogbeat data and continue to use the pre-built dashboards.
I configured logstash to listen for the beats output with no issue, and I have a conditional elasticsearch output block to send the winlogbeat data into its own index like this:

input {
     beats {
           port => 5044
		   type => winlogbeat
        }
...
if [type] == "winlogbeat" {
	            elasticsearch {
	              hosts => ["https://localhost:9200"]	
				  index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

and that works fine. The issue seems to be when I go to the pre-built dashboards I get a failed shards message for the new winlogbeat index.

I guess my question is how can I make the logstash winlogbeat input match the winlogbeat input that goes directly to elasticsearch so that these dashboards continue to work?

Thanks!
Mark

You need to configure the output as the example in the documentation so it will also use the ingest pipeline in Elasticsearch.

Hi Leandro! That pointed me in the right direction, although it differed from the documentation. I did have to load the ingest pipelines using this command

winlogbeat setup --pipelines
not 
winlogbeat setup --pipelines --modules sysmon,security

but then the inserts from logstash to elasticsearch would fail so I had to add this filter in logstash to choose the correct pipeline

#filter for winlogbeat
 if [type] == "winlogbeat"{
    mutate { add_field => { "[@metadata][pipeline]" => "winlogbeat-%{[agent][version]}-routing" } }
  } else if !([@metadata][pipeline]) {
    mutate { add_field => { "[@metadata][pipeline]" => "" } }
  }  
#end filter for winlogbeat

Now the dashboards are working and logstash is receiving the data like it should.

Thanks!
Mark

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.