i am interested in adding a new field 'status' based on the 'endtime' column value, i.e if its not 'null' then the status should be completed else open.
also need to calculate the timedifference between 'endtime' and 'createtime' if the endtime value is not 'null' and then add a new field 'timetaken'
I tried the changes, but still i'm not able to index the fileds with null value as endtime..
the output section is as below
output {
elasticsearch {
hosts => ["localhost:9200"]
document_id => "%{productname}-%{pid}"
index => "records"
}
}
[2019-08-22T10:01:42,774][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"Express product-pid", :_index=>"records", :_type=>"doc", :_routing=>nil}, #LogStash::Event:0x2cdf70c4], :response=>{"index"=>{"_index"=>"records", "_type"=>"doc", "_id"=>"Express product-pid", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [endtime]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "NULL""}}}}}
from the "reason"=>"Invalid format: "NULL"" , its surely because of null value, but i need this to be indexed to the same index "records"
can you please validate
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.