# Not able to create index based on log date field **URL:** \ **Category:** Logstash\ **Created:** [October 12, 2015, 9:47pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042 "2015-10-12T21:47:21Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 12, 2015, 9:47pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/1 "2015-10-12T21:47:21Z")
Hi Experts, I want to create an index based on a log date field , my logs in csv file are like logtime,name 10/13/2015 2:30,vg my LS conf has filter { csv { columns =\> ["logtime","name"] separator =\> "," } } filter { date { match =\> ["logtime","M/d/yyyy h:mm"] target =\> "logtime" } } now after running this I am getting an error ←[33mFailed parsing date from field {:field=\>"logtime", :value=\>"logtime", :exception=\>java.lang.IllegalArgumentException: In valid format: "logtime", :level=\>:warn}←[0m Please help me to understand what is the problem ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 12, 2015, 9:50pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/2 "2015-10-12T21:50:09Z")
Shouldn't `logtime` be `MM/dd/yyyy h:mm`? ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 12, 2015, 9:56pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/3 "2015-10-12T21:56:39Z")
Sorry Mark in my post I mentioned "M/d/yyyy h:mm". Even with "MM/dd/yyyy h:mm" it's giving same error , though Index has been created but I do not see logtime field in the selection , I can still see @timestamp field . My logtime field type is still string , I guess this is the problem ![](https://us1.discourse-cdn.com/elastic/original/2X/5/5c7b36d4af53ef979f92f5da4251b71d3f4b5bcb.png) ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 12, 2015, 10:26pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/4 "2015-10-12T22:26:27Z")
Moved 1 step ahead , it seems it's working even with an error . First I have created a template for this index to define date type explicitly for this filed (logtime).Now I can see both the fields in the selection , so now I have created index based on logtime .Any Idea why I am still getting this error ? Error ![](https://us1.discourse-cdn.com/elastic/original/2X/1/1ec44f956f56e70be92ef0b360fc18d885217f61.png) Kibana Index ![](https://us1.discourse-cdn.com/elastic/original/2X/3/35aab2b8f7ccccb5305ef68c2d16b036ae8d8078.png) ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 12, 2015, 10:53pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/5 "2015-10-12T22:53:22Z")
Check your ES logs, it should mentioned something about the 400 error. ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 12, 2015, 10:59pm UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/6 "2015-10-12T22:59:00Z")
It's full with error messages and some of the portion says Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [logtime], tried both date format [dateOptionalTime], and timestamp number with locale [] at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:617) at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:535) at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:239) at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:401) ... 13 more Caused by: java.lang.IllegalArgumentException: Invalid format: "logtime" at org.elasticsearch.common.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780) at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:612) ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 13, 2015, 12:16am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/7 "2015-10-13T00:16:43Z")
Nope !!! no luck yet , I tried many options but still same error . It seems except from default format ES does not support any other date format . ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 13, 2015, 6:09am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/8 "2015-10-13T06:09:09Z")
Take ES out of the equation and use a simple `stdout { codec => rubydebug }` output to verify that your `logtime` fields has the expected contents. In the screenshot above showing the "failed action with response of 400" error message I can also see an error message indicating that the date parsing failed. This would leave `logtime` untouched which explains why ES has problems parsing the date. ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 13, 2015, 7:09am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/9 "2015-10-13T07:09:22Z")
This is what I got after using stdout only . I am using date format as **LS Filter** filter { date { match =\> ["logtime","yyyy-MM-dd HH:mm:ss"] target =\> "logtime" } } **My Data** logtime,name 2015-10-13 04:10:05,vg **Error** ←[33mFailed parsing date from field {:field=\>"logtime", :value=\>"logtime", :exception=\>java.lang.IllegalArgumentException: In valid format: "logtime", :level=\>:warn}←[0m { "message" =\> [ [0] "logtime,name\r" ], "@version" =\> "1", "@timestamp" =\> "2015-10-13T07:13:15.271Z", "host" =\> "LP-54EE752450D4", "path" =\> "E:\ELK\_Traning\logstash-1.5.1\bin\time.csv", "logtime" =\> "logtime", "name" =\> "name" } { "message" =\> [ [0] "2015-10-13 04:10:05,vg\r" ], "@version" =\> "1", "@timestamp" =\> "2015-10-13T07:13:15.271Z", "host" =\> "LP-54EE752450D4", "path" =\> "E:\ELK\_Traning\logstash-1.5.1\bin\time.csv", "logtime" =\> "2015-10-12T22:40:05.000Z", "name" =\> "vg" } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 13, 2015, 7:34am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/10 "2015-10-13T07:34:20Z")
Well, if you try to parse the header line ("logtime,name") things are obviously not going to work. You should drop those lines, perhaps by checking if the message begins with what looks like a field name rather than a date (to allow for renames of the fields). Something like this? ``` filter { if [message] =~ /^[a-z]*,/ { drop { } } ... } ``` ---
**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Post date:** [October 13, 2015, 7:52am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/11 "2015-10-13T07:52:00Z")
It works like Charm , thank you Mangnus I should have catch this earlier as Error log directly says that value for that field is a string which causes the issue .Thanks again for your support. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:26am UTC](https://discuss.elastic.co/t/not-able-to-create-index-based-on-log-date-field/32042/12 "2017-07-06T05:26:44Z")