Hi,
I am using a setup where Metricbeat running in remote server sends data to Logstash.
Metricbeat sends tons of data to Logstash and I don't want everything to be stored by Elasticsearch.
At each interval, Logstash receives from Metricbeat as below which are in 100s of different event.
[2017-05-11T02:30:46,633][DEBUG][logstash.pipeline ] output received {"event"=>{"@timestamp"=>2017-05-11T06:20:49.399Z, "system"=>{"process"=>{"memory"=>{"rss"=>{"pct"=>0.0, "bytes"=>0}, "size"=>0, "share"=>0}, "pgid"=>0, "name"=>"scsi_eh_3", "cpu"=>{"start_time"=>"2017-02-23T18:26:49.000Z", "total"=>{"pct"=>0.0}}, "pid"=>657, "state"=>"sleeping", "fd"=>{"limit"=>{"hard"=>4096, "soft"=>1024}, "open"=>0}, "username"=>"root", "ppid"=>2}}, "beat"=>{"hostname"=>"pte401", "name"=>"pte401", "version"=>"5.3.2"}, "@version"=>"1", "host"=>"pte401", "metricset"=>{"rtt"=>218091, "module"=>"system", "name"=>"process"}, "type"=>"metricsets", "tags"=>["beats_input_raw_event"]}}
I don't want _"cpu"=>{"start_time"} to be stored. Let's take it that is of no use for me.
How do I remove it from Logstash filter, so that it is not stored in Elasticsearch? Can you please help?
I tried to use as below, but it didn't help. I see it in Kibana still there.
mutate {
remove_field => [ "system.process.memory.share", "system.process.cpu.start_time" ]
}