Field name [server.example.com] cannot contain '.'

Elastic Stack Logstash
1

I have an application that sends JSON messages to logstash. The JSON looks like this:

{
    "status": "Successful",
    "hosts": {
        "server.example.com": {
            "ok": 2,
            "failed": "false"
        },
        "server2.example.com": {
           "ok": 1,
           "failed": "true"
        }
    }
}

Elasticsearch throws the error 'Field name [server.example.com] cannot contain '.'"'

I tried adding de_dot filter to replace the dots, but I can't get the nesting to work properly. It adds the tag, but the fields all stay the same.

de_dot {
    fields => [ "hosts" ]
    add_tag => [ "de_dot" ]
    nested => true
}

Any suggestions on what I'm doing wrong with de_dot (or better options!)?

logstash 2.4.0
elasticsearch 2.4.0

2

I was able to convert nested fqdn strings to short names with this incredibly unappealing solution. If anyone has a better alternative I'd love to hear them!

ruby {
    code => "
        data = event['hosts'].clone.to_hash;
        data.each do |k,v|
            newFieldName = k.split('.')[0]
            event['hosts'].delete(k)
            event['hosts'][newFieldName] = v
        end
    "
}

Thanks,
Ryan

3

It seems the de_dot filter only considers top-level fields. So yes, for now your ruby filter workaround is probably your best bet.