I'm trying to output to a very picky HTTP API, and the existence of a field in the JSON that is posted causes it confusion. I originally did the integration work with 5.0.0 (latest at the time) but then when we went to deploy, grabbed the latest version (5.1.0) I then checked every iteration back to 5.0.0 and found that the ability to remove the 'tags' field broke in 5.0.1 (when oddly, there was some work done around how the tags field is handled.)
[ronnocol@tarpon logstash-5.0.0]$ bin/logstash -e 'filter { mutate { remove_field => ["tags"] }}'
Sending Logstash logs to /home/loconnor/projects/logstash-5.0.0/logs which is now configured via log4j2.properties.
The stdin plugin is now waiting for input:
[2016-12-15T09:52:02,658][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2016-12-15T09:52:02,675][INFO ][logstash.pipeline ] Pipeline main started
[2016-12-15T09:52:02,727][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
lsdkfj
{
"@timestamp" => 2016-12-15T17:52:07.282Z,
"@version" => "1",
"host" => "tarpon.engr.blah.com",
"message" => "lsdkfj",
"type" => "stdin"
}
See, perfect, no tags field...
In 5.0.1 (and above) however:
[loconnor@tarpon logstash-5.0.1]$ bin/logstash -e 'filter { mutate { remove_field => ["tags"] }}'
Sending Logstash's logs to /home/loconnor/projects/logstash-5.0.1/logs which is now configured via log4j2.properties
The stdin plugin is now waiting for input:
[2016-12-15T10:30:44,329][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>1000}
[2016-12-15T10:30:44,344][INFO ][logstash.pipeline ] Pipeline main started
[2016-12-15T10:30:44,390][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
asdf
{
"@timestamp" => 2016-12-15T18:30:50.053Z,
"@version" => "1",
"host" => "tarpon.engr.blah.com",
"message" => "asdf",
"type" => "stdin",
"tags" => []
}
[2016-12-15T10:30:54,365][WARN ][logstash.agent ] stopping pipeline {:id=>"main"}