# \[Resolved\]Logstash - Question for how to use if condition to skip invalid date format **URL:** \ **Category:** Logstash\ **Created:** [March 29, 2019, 5:54am UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472 "2019-03-29T05:54:07Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [March 29, 2019, 5:54am UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472/1 "2019-03-29T05:54:07Z")
Hi Team, I have a CSV file which has a column named 'COMMIT\_DATE', the values of this column input by end user sometimes are a string , instead of a date. For example, there's invalid date value 'test' in below source data. ``` GIT_ORG,GIT_REPOS,COMMIT_SHA1,COMMIT_AUTHOR,COMMIT_DATE bizx,au-recruiting,6739c82bcf830b05d4d36e9fd715bc5715e0c380,Kaderjan Ilghar,2018-01-24 bizx,au-V4,72db50e50121110e98ccb18d90b47f227df96ea1,csong,test bizx,idl-analytics-api,1be44f52f25f6b540f284eb17e8cee5838826cb9,ssheriff,2017-03-30 bizx,au-employeecentral,23d7a081034c2a27b43686ab9a3468774b03e07d,Felix Roth,test ``` I want logstash not process and skip the records that contains invalid date format with applying the if condition. Here is the logstash.conf but it doesn't work as expected. Can you help how to write the right if condition? ``` filter { if [COMMIT_DATE] == "yyyy-MM-dd" { csv { columns => [ "GIT_ORG", "GIT_REPOS", "COMMIT_SHA1", "COMMIT_AUTHOR", "COMMIT_DATE" ] separator => "," skip_header => "true" } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 29, 2019, 12:34pm UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472/2 "2019-03-29T12:34:45Z")
I would use a date filter to parse COMMIT\_DATE and then ``` if _dateparsefailure in [tags] { drop {} } ``` ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [April 1, 2019, 1:58am UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472/3 "2019-04-01T01:58:04Z")
Badger -Error is returned with running below configuration, doesn't understand what format error it means. ``` [2019-04-01T10:27:32,953][ERROR][logstash.outputs.elasticsearch] Failed to install template. {:message=>"Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])\"{\n \"template\": \"test1\",\n \"index_patterns\":[\"test1\"],\n \"settings\": {\n \"number_of_shards\": 1,\n\t\"number_of_replicas\" : 1,\n\t\"refresh_interval\": \"5s\" \n },\n\n \"mappings\": {\n \"doc\": {\n \"dynamic_templates\": \n {\ n \"match\": \"*\",\n \"match_mapping_type\": \"*\",\n \"mapping\": {\n \"type\": \"keyword\"\n }\n }\n }\n }\n\n\"; line: 1, column: 1])\n at [Source: (byte[])\"{\n \"templ ate\": \"test1\",\n \"index_patterns\":[\"test1\"],\n \"settings\": {\n \"number_of_shards\": 1,\n\t\"number_of_replicas\" : 1,\n\t\"refresh_interval\": \"5s\" \n },\n\n \"mappings\": {\n \"doc\": {\n \"dynamic_temp lates\": \n {\n \"match\": \"*\",\n \"match_mapping_type\": \"*\",\n \"mapping\": {\n \"type\": \"keyword\"\n }\n }\n }\n }\n\n\"; line: 23, column: 369]", :class=>"LogS tash::Json::ParserError", :backtrace=>["C:/elkstack/logstash-6.5.1/logstash-6.5.1/logstash-core/lib/logstash/json.rb:16:in `jruby_load'", "C:/elkstack/logstash-6.5.1/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-el asticsearch-9.2.1-java/lib/logstash/outputs/elasticsearch/template_manager.rb:33:in `read_template_file'", "C:/elkstack/logstash-6.5.1/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.1-java/lib/logs tash/outputs/elasticsearch/template_manager.rb:17:in `get_template'", "C:/elkstack/logstash-6.5.1/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.1-java/lib/logstash/outputs/elasticsearch/template_m anager.rb:7:in `install_template'", "C:/elkstack/logstash-6.5.1/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.1-java/lib/logstash/outputs/elasticsearch/common.rb:118:in `install_template'", "C:/el kstack/logstash-6.5.1/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.2.1-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in install_template_after_successful_connection'"]} [2019-04-01T10:27:33,519][TRACE][logstash.inputs.file] Registering file input {:path=>["C:/elkstack/elasticsearch-6.5.1/logs/test1.csv"]} ``` logstash configuraiton ``` filter { date { match => ["COMMIT_DATE", "yyyy-MM-dd"] target => "COMMIT_DATE" } if "_dateparsefailure" in [tags] { drop {} } csv { columns => [ "GIT_ORG", "GIT_REPOS", "COMMIT_SHA1", "COMMIT_AUTHOR", "COMMIT_DATE" ] separator => "," skip_header => "true" } } ``` ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [April 1, 2019, 5:39am UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472/4 "2019-04-01T05:39:07Z")
I figure out the error is in index template. Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 29, 2019, 5:39am UTC](https://discuss.elastic.co/t/resolved-logstash-question-for-how-to-use-if-condition-to-skip-invalid-date-format/174472/5 "2019-04-29T05:39:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.