# Logstash how to handle exceptions **URL:** \ **Category:** Logstash\ **Created:** [December 23, 2015, 10:18am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843 "2015-12-23T10:18:39Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [December 23, 2015, 10:18am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/1 "2015-12-23T10:18:39Z")
Hi, i have a problem in managing exceptions in filter logstash. I have to define some dates which don't always show the same format and this causes the following error: Ruby exception occurred: invalid strptime format - `%d/%m/%Y %H.%M.%S %Z' {:level=\>:error} The logstash filter is : if [DATA] { ruby { code =\> " event['DATA'] = Time.strptime(event['DATA']+' Europe/Rome', '%d/%m/%Y %H.%M.%S %Z').gmtime.strftime('%Y-%m-%dT%H:%M:%S.%LZ') " } } Not only it does return an exception, but sometimes Logstash doesn't load data on Elastic Search anymore. How i can handle the exception ? thank you ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 23, 2015, 10:51am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/2 "2015-12-23T10:51:57Z")
Would you not be able to use the [date filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-match) with an array of possible date formats? ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [December 23, 2015, 11:09am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/3 "2015-12-23T11:09:18Z")
the format could be different each time and it could be difficult to expect all of them ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [December 23, 2015, 1:44pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/4 "2015-12-23T13:44:20Z")
Hi , thanks for the advice, could you please show me for instance how to apply the date filter supposing that the format could be '%d/%m/%Y %H.%M.%S %Z' '%d/%m/%Y %H:%M:%S %Z' thank you ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 23, 2015, 1:50pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/5 "2015-12-23T13:50:33Z")
If you have multiple possible separators, you can try to normalize them using a mutate gsub filter. This will reduce the number of patterns you need to match. Can you provide actual exemples of the date panterns you want to match? ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [December 23, 2015, 1:57pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/6 "2015-12-23T13:57:56Z")
the dates are for example: 18/12/2015 17.53.05 or 18/12/2015 17:53:05 and the expected result is: 2015-12-18 T17:53:05.000Z ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 23, 2015, 3:52pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/7 "2015-12-23T15:52:47Z")
What have you tried so far? ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [December 23, 2015, 4:17pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/8 "2015-12-23T16:17:08Z")
the gsub solution doesn't seem to be working. i've tried to use date filter but it doesn't work either. maybe i am doing something wrong ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 23, 2015, 10:07pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/9 "2015-12-23T22:07:33Z")
> [@chicco\_95](#): > > '%d/%m/%Y %H.%M.%S %Z''%d/%m/%Y %H:%M:%S %Z' Per [the docs](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-match) you should be able to use something like; `match => ["datefield", "d/m/Y H.M.S Z", "d/m/Y H.M:S Z"]` ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 9:15am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/10 "2016-01-04T09:15:04Z")
Sorry, but it doesn't work either. this is my csv input : ID ; DATE TTM000005718043 ; 11/12/2015 09.42.12; TTM000006099454 ; 11/12/2015 12:06:08; TTM000006097855 ; 11/12/2015 13.22.56; TTM000001111111 ; 11/12/2015 13:24:14; thisi is my filter of config file for logstash: filter { ``` if [type] == "test" { csv { columns => ["ID","DATE"] separator => ";" } date { match => ["DATE", "d/m/Y H.M.S Z", "d/m/Y H.M:S Z", "ISO8601"] } } ``` } where I'm wrong? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2016, 10:22am UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/11 "2016-01-04T10:22:12Z")
Try: ``` match => ["DATE", "dd/MM/YYYY HH.mm.ss Z", "dd/MM/YYYY HH.mm.ss Z", "ISO8601"] ``` ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 12:59pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/12 "2016-01-04T12:59:36Z")
The format is the same. I HAVE CHANGED THE FORMAT , BUT DOES NOT WORK .... THE FORMAT IS THE SAME OF ENTRY ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2016, 1:34pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/13 "2016-01-04T13:34:08Z")
Since you don't have a timezone at the end remove the " Z" too. Then it works with the example you provided: ```auto $ cat test.config input { stdin { } } output { stdout { codec => rubydebug } } filter { date { match => ["message", "dd/MM/YYYY HH.mm.ss"] } } $ echo '11/12/2015 09.42.12' | /opt/logstash/bin/logstash -f test.config Logstash startup completed { "message" => "11/12/2015 09.42.12", "@version" => "1", "@timestamp" => "2015-12-11T08:42:12.000Z", "host" => "lnxolofon" } Logstash shutdown completed ``` ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 1:44pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/14 "2016-01-04T13:44:36Z")
but from this format 11/12/2015 09.42.12 must become this format 2015-12-11T09:42:12.000Z I tried to remove the time zone but does not change the situation ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2016, 1:57pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/15 "2016-01-04T13:57:19Z")
If your timestamp already is in UTC and you don't want the date filter to interpret them as local time, use the filter's `timezone` option: ```auto date { ... timezone => "UTC" } ``` ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 2:18pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/16 "2016-01-04T14:18:52Z")
Nothing changes.... sorry but I do not understand , I try to re-explain... these are the input record of my csv: ID ; DATE TTM000005718043;2015/11/12 09.42.12; TTM000006099454;2015/11/12 00:06:08; TTM000006097855;2015/11/12 13.22.56; TTM000001111111;2015/11/12 13:24:14; thisi is my config file for logstash: input { file { path =\> "input/test\_\*.csv" start\_position =\> "beginning" type =\> "test" sincedb\_path =\> "work/.sincedb\_test" codec =\> plain { charset =\> "ISO-8859-1" } } } filter { if [type] == "test" { csv { columns =\> ["ID","DATE"] separator =\> ";" } date { match =\> ["DATE", "dd/MM/YYYY HH.mm.ss", "dd/MM/YYYY HH:mm:ss", "ISO8601"] } mutate{ add\_field =\> ["key", "%{ID}"] } } } output { elasticsearch { host =\> "my\_host" cluster =\> "my\_cluster" index =\> "my\_index" document\_type =\> "%{type}" document\_id =\> "%{key}" template =\> "config/template.json" } } This is the output I want on ES: TTM000005718043; 2015-12-11T09:42:12.000Z TTM000006099454; 2015-12-11T00:06:08.000Z TTM000006097855; 2015-12-11T13:22:56.000Z TTM000001111111; 2015-12-11T13:24:14.000Z Where is the error? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 4, 2016, 2:42pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/17 "2016-01-04T14:42:24Z")
We have your configuration and the expected result, but what is the current result from your configuration? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 4, 2016, 2:44pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/18 "2016-01-04T14:44:48Z")
You seem to have a space a the start of the date column, which your date pattern does not seem to account for. You can remove this through a [mutate strip](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-strip) filter. I would also recommend using the stdout filter with ruby debug codec while troubleshooting this issue. ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 2:45pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/19 "2016-01-04T14:45:47Z")
yes , sorry.... This is the current result: "ID" =\> "TTM000001111111", "DATE" =\> "11/12/2015 13:24:14" ---
**Author:** ![chicco\_95](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@chicco\_95](https://discuss.elastic.co/u/chicco_95)\ **Post date:** [January 4, 2016, 2:48pm UTC](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843/20 "2016-01-04T14:48:00Z")
the space is not there, I was wrong to bring the content. and i already use a rubydebug [Next page](https://discuss.elastic.co/t/logstash-how-to-handle-exceptions/37843.md?page=2)