Dots in Fields: For those of you who have not yet upgraded to Logstash 2.0 and Elasticsearch 2.0, please be advised that Elasticsearch 2.0 does not allow field names to contain the . character (further details here). This has caused some grief with many users who are upgrading and finding that configurations which did work, no longer do.
Some plugins already have been updated to compensate for this breaking change, including logstash-filter-metrics and logstash-filter-elapsed. These plugin updates are available for Logstash 2.0. To upgrade to the latest version of these plugins, the command is:
bin/plugin update logstash-PLUGINTYPE-NAME
Where PLUGINTYPE can be input, filter, output, or codec, and NAME is the name of the plugin.
Although the de-dotting process is better done upstream, we understand there may be scenarios where users may have no control over the source field names. Therefor, a de_dot filter is being developed to allow users to rename fields to not have dots in them. The replacement separator will be configurable. Please be patient while this is written, reviewed and tested. An announcement of its availability will be made here when it is ready.
Mapping changes: Users may have custom template changes, so by default a Logstash upgrade will leave the template as is. Even if you don’t have a custom template, Logstash will not overwrite an existing template by default.
There is one known issue (removal of path) with using GeoIP filter that needs a manual update to the template.
Note: If you have custom template changes, please make sure to save it and merge any changes. You can get the existing template by running:
curl -XGET localhost:9200/_template/logstash
Add the following option to your Logstash config:
output {
elasticsearch {
template_overwrite => true
}
}
Restart Logstash.
Please feel free to reply here if you run into any issues or have any suggestions!