Making sense of logstash error logs

Elastic Stack Logstash
1

Here is logstash.log file when restarting logstash and loading my csv file. It appears to load fine and I can view the contents in Kibana but I can't figure out this error message.

`

, "CVSS"=>"CVSS", "Severity"=>"Severity", "High"=>0.0, "Medium"=>0.0, "Low"=>0.0, "Log"=>0.0, "False Positive"=>0.0, "Total"=>0.0}, "type"]}>>], :response=>{"create"=>{"_index"=>"vuln", "_type"=>"logs", "_id"=>"AV4pMHwZuPh7f-_JCZDp", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper [Scan End] of different type, current_type [date], merged_type [string]"}}}, :level=>:warn}

`

csv file:

IP,Hostname,OS,Scan Start,Scan End,CVSS,Severity,High,Medium,Low,Log,False Positive,Total
10.100.1.4,,cpe:/o:cisco,2017-06-16T18:50:26Z,2017-06-16T19:15:35Z,10.0,High,1,5,1,17,0,25

data.conf:

> input {
>   file {
>     path => "/home/user/reports/report1.csv"
>     start_position => "beginning"
>     # to read from the beginning of file
>   }
> }
> 
> filter {
>     csv {
>         separator => ","
>         columns => ["IP", "Hostname", "OS", "Scan Start", "Scan End", "CVSS", "Severity", "High", "Medium", "Low", "Log", "False Positive", "Total"]
>         
>     }
>     mutate {
>         convert => { "High" => "float" }
>         convert => { "Medium" => "float" }
>         convert => { "Low" => "float" }
>         convert => { "Log" => "float" }
>         convert => { "False Positive" => "float" }
>         convert => { "Total" => "float" }
>     }
> }
> 
> output {
>    elasticsearch {
>      action => "index"
>      hosts => "ip:port"
>      index => "vuln"
>      workers => 1
>    }
> stdout {}
> }
2

That error looks like it is coming from Elasticsearch, not logstash. My best guess is that the Elasticsearch mapping has [Scan End] as a data type of string, and you are trying to insert it as a datetime. Or vice-versa.
You may have to explicitly convert [Scan End] as a datetime in logstash and then manually update the mapping in Elasticsearch to be a datetime.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.