Two records written by Logstash for each log line


(Vivek Sharma) #1

Hi

The ELK setup which I am using is:
Filebeat -> Logstash -> Elasticsearch -> Kibana

Filebeat configuration file is as below:

filebeat.prospectors:

-- input_type: log
paths:
- C:\ELK\log*.log
fields:
tech_stack: XXX
kpi_type: YYY

The logstash configuration file is as below:

> input {
>   beats {
>     port => 5044
>   }
>    
> }
> 
>   filter {
>   grok {
>     match => { "message" => [

> 	"%{SYSLOGTIMESTAMP:timestamp} \[Host:%{HOSTNAME:host_name}\:-1\|Service:%{PROG:log_service_name}\|Context:%{USERNAME:context}\|Session:%{USERNAME:session_id}\|User:%{USERNAME:user_name}\|ApplicationId:%{USERNAME:application_id}\|MessageId:%{USERNAME:message_id}\|"
> 	] }
>   }
> 	
>     mutate {
>       add_field => { "tech_stack" => "%{[fields][tech_stack]}" }
> 	  add_field => { "kpi_type" => "%{[fields][kpi_type]}" }
>     }
> }
> 
> output {
>   elasticsearch {
>     hosts => "localhost:9200"
> 	user => "elastic"
> 	password => "changeme"
>     manage_template => false
> 	index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" 
>     document_type => "%{[@metadata][type]}" 
>   }
>    stdout { codec => rubydebug }
> }

Now the issue is that, when I insert a record in log file, then two records are displayed for each line of log that I am inserting.

Can anyone please help.

Thanks


(Jim Unger) #2

I'm not sure what would be causing this, but you might want to try posting this question in the logstash discuss forum.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.