Invalid format (illegal_argument_exception)

Elastic Stack Logstash
1

Can someone help me understand the underlying issue here. I am using kv filter to parse URL query parameters and pushing them to dynamically named indexes.

input {
  beats {
    port => 5045
  }
}

filter {
    grok { .. }
    kv {
        source => "apache2.access.url"
        field_split => "&?"
    }
}
output {
  elasticsearch {
    index => "httpd-%{[fields][app]}"
    ...
  }

Is "from" a reserved field, where's the format defined in this case ? Are there any illegal characters in the string ? Or am I missing the picture entirely ?

Any help would be appreciated.

[2017-12-19T02:06:59,816][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"httpd-towerportal-2017.12.19", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x9323d8b], :response=>{"index"=>{"_index"=>"httpd-xxx-2017.12.19", "_type"=>"doc", "_id"=>"***", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [from]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "/corporate-directory/""}}}}}

2

The underlying issue here was elasticsearch's automatic mapping identifying "from" as date because it was the case for few query parameters which were getting pushed before this one.

An index template specifying the type fixed the issue.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.