# Date\_time\_parse\_exception Warning message in Logstash while fetching data from CSV file **URL:** \ **Category:** Logstash\ **Created:** [November 10, 2020, 2:24pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922 "2020-11-10T14:24:36Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Pavitra\_Poojary](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavitra_poojary/32/77753_2.png) [@Pavitra\_Poojary](https://discuss.elastic.co/u/Pavitra_Poojary)\ **Post date:** [November 10, 2020, 2:24pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/1 "2020-11-10T14:24:36Z")
Hi, I am trying to fetch data from few csv file ``` input { file { path => "/root/API*" start_position => "beginning" sincedb_path => "/dev/null" type => "API" } } filter { csv { separator => "," columns => ["TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP","URI_ID_DERIVED"] } } output { elasticsearch { hosts => ["xxx:443"] //443 because of managed ES index => "apisalesforceapi-%{+YYYY.MM}" user => "xxx" password => "xxx" ilm_enabled => false //Managed ES } stdout { codec => rubydebug } } ``` Here : "TIMESTAMP\_DERIVED" -\> is the field which is causing the issue. Logstash is able to read the csv file and index gets created in Managed ES as well, however i end up getting the below WARN everytime `[2020-11-10T14:04:12,995][WARN][logstash.outputs.elasticsearch][python][f1948bfb7238388c36f40fa69ae4943193b1e7dcc21fa07b7f74a47b7a0c1474] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"apisalesforceapi-2020.11", :routing=>nil, :_type=>"_doc"}, #], :response=>{"index"=>{"_index"=>"apisalesforceapi-2020.11", "_type"=>"_doc", "_id"=>"44N4snUBvPvlSgV_p801", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [TIMESTAMP_DERIVED] of type [date] in document with id '44N4snUBvPvlSgV_p801'. Preview of field's value: 'TIMESTAMP_DERIVED'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [TIMESTAMP_DERIVED] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}` Please help me resolve this !! Thank you !! ---
**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\ **Post date:** [November 10, 2020, 2:40pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/2 "2020-11-10T14:40:34Z")
Can you share an example of lines of your input log ? here the error means you need to apply a date filter to your date so it can be parsed correctly ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 10, 2020, 3:31pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/3 "2020-11-10T15:31:06Z")
> [@Pavitra\_Poojary](#): > > "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field [TIMESTAMP\_DERIVED] of type [date] in document with id '44N4snUBvPvlSgV\_p801'. Preview of field's value: 'TIMESTAMP\_DERIVED'" You have a header row. elasticsearch expects TIMESTAMP\_DERIVED to be a date and it cannot parse the string value TIMESTAMP\_DERIVED as a date. You might find the skip\_header option on the csv filter helpful. ---
**Author:** ![Pavitra\_Poojary](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavitra_poojary/32/77753_2.png) [@Pavitra\_Poojary](https://discuss.elastic.co/u/Pavitra_Poojary)\ **Post date:** [November 10, 2020, 5:27pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/4 "2020-11-10T17:27:34Z")
hi, this is my csv file ``` "EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","RUN_TIME","CPU_TIME","URI","SESSION_KEY","LOGIN_KEY","REQUEST_STATUS","DB_TOTAL_TIME","API_TYPE","API_VERSION","CLIENT_NAME","METHOD_NAME","ENTITY_NAME","ROWS_PROCESSED","REQUEST_SIZE","RESPONSE_SIZE","DB_BLOCKS","DB_CPU_TIME","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP","URI_ID_DERIVED" "API","20201108061648.336","4Zxx--","00xxx","00xxx","459xx","31","Api","Wbsxxx","hfRxxx","","32xxx","M","50.0","sfdx xxx","meta_retrieve","","","843","330","590","30","2020-11-08T06:16:48.336Z","0053Jxxx","103.xx.xx.xxx","" ``` above example i removed the column names as it will be more. Issue seems only with TIMESTAMP\_DERIVED ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 10, 2020, 5:48pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/5 "2020-11-10T17:48:10Z")
The other fields are strings, so elasticsearch will not care if it gets a value like "REQUEST\_ID". date fields do care. ---
**Author:** ![Pavitra\_Poojary](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavitra_poojary/32/77753_2.png) [@Pavitra\_Poojary](https://discuss.elastic.co/u/Pavitra_Poojary)\ **Post date:** [November 10, 2020, 5:54pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/6 "2020-11-10T17:54:11Z")
Yes adding skip\_header in the filter part resolved the issue ``` csv { separator => "," columns => ["EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","RUN_TIME","CPU_TIME","URI","SESSION_KEY","LOGIN_KEY","REQUEST_STATUS","DB_TOTAL_TIME","API_TYPE","API_VERSION","CLIENT_NAME","METHOD_NAME","ENTITY_NAME","ROWS_PROCESSED","REQUEST_SIZE","RESPONSE_SIZE","DB_BLOCKS","DB_CPU_TIME","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP","URI_ID_DERIVED"] skip_header => "true" } } ``` Thank you so much !! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 8, 2020, 5:54pm UTC](https://discuss.elastic.co/t/date-time-parse-exception-warning-message-in-logstash-while-fetching-data-from-csv-file/254922/7 "2020-12-08T17:54:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.