Logstash filter - error

Elastic Stack Logstash
1

Hi team, i am new on ELK stack, and i was trying to read log (given below) through filter, kindly help to write the filter on the below log,

2018-08-25 14:14:29,026 [http-nio-8085-exec-458] INFO com..cdb.utils.Utils - getHTTPClientResponse -sUrl : https://vender:443/v1/services/url?lineOfBusiness=Mobility

2018-08-25 14:14:29,026 [http-nio-8085-exec-458] INFO com..cdb.utils.Utils - getHTTPClientResponse -Service IP Address : 10.0.0.1
2018-08-25 14:14:29,027 [http-nio-8085-exec-458] INFO com..cdb.utils.Utils - getHTTPClientResponse -Service Port (https): 443

2018-08-25 14:14:29,027 [http-nio-8085-exec-458] INFO com..cdb.utils.Utils - getHTTPClientResponse -Final URL: https://vender:443/v1/services/url?lineOfBusiness=Mobility

2018-08-25 14:14:29,061 [http-nio-8085-exec-458] INFO com..cdb.utils.Utils - getHTTPClientResponse -Total time consumed after execute Method34

2018-08-25 14:14:29,061 [http-nio-8085-exec-458] INFO com..cdb.responsetype.CDBMobilityResponse - loadObject -

2018-08-25 14:14:29,061 [http-nio-8085-exec-458] INFO com..cdb.responsetype.CDBMobilityResponse - loadObject -HTTPStatus Code from third Party-> 200

the filter which i was running is below

input {
file
{
path => "/opt/logstash-6.5.4/data/CDBOptimus.log"
start_position => "beginning"
}
}

The filter part of this file is commented out to indicate that it is

optional.

filter {
date {
match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS" ]
target => "logdate"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "CDB-Optimus"
}
}

and the error which i got is (in logstash log)

[2019-01-02T05:43:27,643][ERROR][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"CDB-Optimus", :_type=>"doc", :routing=>nil}, #LogStash::Event:0x19aef96a], :response=>{"index"=>{"_index"=>"CDB-Optimus", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [CDB-Optimus], must be lowercase", "index_uuid"=>"na", "index"=>"CDB-Optimus"}}}}

2

The error message indicates you are using upper case letters in your index name, that is not allowed. Fix that first and then use the grok filter to generate a pattern that matches your log lines.

3

thanks after making them in lower case, i got the exact error and i got the filter created based on the above log, thanks Andreas!!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.