Date pattern specified is mapping not applied


(Ram Viswanadha) #1

I have events coming in that have interesting date formats. I set up the
pattern to be accepted in the mapping of the index. But, I am getting the
following errors
curl -XPOST
'http://localhost:9200/events_addon_20130925/topbar?pretty=true' -d
'{"url": "http://money-on-mobile.net/index.aspx", "install_date": "Sun Sep
15 2013 16:41:53 GMT+0530 (India Standard Time)", "state": "close", "bfp":
"1008244289", "event_date": "Sun Sep 22 2013 20:03:46 GMT+0530 (India
Standard Time)", "browser": "chrome"}'
{
"error" : "MapperParsingException[failed to parse [install_date]];
nested: MapperParsingException[failed to parse date field [Sun Sep 15 2013
16:41:53 GMT+0530 (India Standard Time)], tried both date format [EEE MMM
dd yyyy HH:mm:ss 'GMT'Z (zzzz)], and timestamp number with locale [null]];
nested: UnsupportedOperationException[Parsing not supported]; ",
"status" : 400
}

Here is the mapping I applied

$ curl -XGET
'http://localhost:9200/events_addon_20130925/topbar/_mapping?pretty=true'
{
"topbar" : {
"properties" : {
"bfp" : {
"type" : "string",
"index" : "not_analyzed",
"omit_norms" : true,
"index_options" : "docs"
},
"browser" : {
"type" : "string"
},
"event_date" : {
"type" : "date",
"format" : "EEE MMM dd yyyy HH:mm:ss 'GMT'Z (zzzz)"
},
"install_date" : {
"type" : "date",
"format" : "EEE MMM dd yyyy HH:mm:ss 'GMT'Z (zzzz)"
},
"state" : {
"type" : "string"
},
"url" : {
"type" : "string"
},
"version_number" : {
"type" : "string",
"index" : "not_analyzed",
"omit_norms" : true,
"index_options" : "docs"
}
}
}
}

I tested the pattern using SimpleDateFormat in Java and it works. Not sure
what I am doing wrong in ElasticSearch. Any idea?

$ JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/ java Test "EEE MMM dd yyyy
HH:mm:ss 'GMT'Z (zzzz)" "Sun Sep 22 2013 20:03:46 GMT+0530 (India Standard
Time)"
The formatted string: Wed Sep 25 2013 21:41:00 GMT+0000 (Coordinated
Universal Time)
The parsed date is: Wed Sep 25 2013 21:41:00 GMT+0000 (Coordinated
Universal Time)
The parsed date from input is: Sun Sep 22 2013 20:03:46 GMT+0530 (India
Standard Time)

import java.text.SimpleDateFormat;
import java.util.Date;

public class Test{
public static void main(String[] args) {
String pattern=args[0];
String dtstr = args[1];
SimpleDateFormat sdf = new SimpleDateFormat(pattern);
Date date = new Date();
try {
//
String fmtd = sdf.format(date);
System.out.println("The formatted string: " +fmtd);
System.out.println("The parsed date is: " +
sdf.format(sdf.parse(fmtd)));
System.out.println("The parsed date from input is: " +
sdf.format(sdf.parse(dtstr)));
} catch (Exception e) {
e.printStackTrace();
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ram Viswanadha) #2

Figured this one out. Joda Time does not support parsing of long timezone
identifiers. So, need to change the data in 6M records :slight_smile:

On Wednesday, September 25, 2013 2:42:25 PM UTC-7, Ram Viswanadha wrote:

I have events coming in that have interesting date formats. I set up the
pattern to be accepted in the mapping of the index. But, I am getting the
following errors
curl -XPOST '
http://localhost:9200/events_addon_20130925/topbar?pretty=true' -d
'{"url": "http://money-on-mobile.net/index.aspx", "install_date": "Sun
Sep 15 2013 16:41:53 GMT+0530 (India Standard Time)", "state": "close",
"bfp": "1008244289", "event_date": "Sun Sep 22 2013 20:03:46 GMT+0530
(India Standard Time)", "browser": "chrome"}'
{
"error" : "MapperParsingException[failed to parse [install_date]];
nested: MapperParsingException[failed to parse date field [Sun Sep 15 2013
16:41:53 GMT+0530 (India Standard Time)], tried both date format [EEE MMM
dd yyyy HH:mm:ss 'GMT'Z (zzzz)], and timestamp number with locale [null]];
nested: UnsupportedOperationException[Parsing not supported]; ",
"status" : 400
}

Here is the mapping I applied

$ curl -XGET '
http://localhost:9200/events_addon_20130925/topbar/_mapping?pretty=true'
{
"topbar" : {
"properties" : {
"bfp" : {
"type" : "string",
"index" : "not_analyzed",
"omit_norms" : true,
"index_options" : "docs"
},
"browser" : {
"type" : "string"
},
"event_date" : {
"type" : "date",
"format" : "EEE MMM dd yyyy HH:mm:ss 'GMT'Z (zzzz)"
},
"install_date" : {
"type" : "date",
"format" : "EEE MMM dd yyyy HH:mm:ss 'GMT'Z (zzzz)"
},
"state" : {
"type" : "string"
},
"url" : {
"type" : "string"
},
"version_number" : {
"type" : "string",
"index" : "not_analyzed",
"omit_norms" : true,
"index_options" : "docs"
}
}
}
}

I tested the pattern using SimpleDateFormat in Java and it works. Not sure
what I am doing wrong in ElasticSearch. Any idea?

$ JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/ java Test "EEE MMM dd yyyy
HH:mm:ss 'GMT'Z (zzzz)" "Sun Sep 22 2013 20:03:46 GMT+0530 (India Standard
Time)"
The formatted string: Wed Sep 25 2013 21:41:00 GMT+0000 (Coordinated
Universal Time)
The parsed date is: Wed Sep 25 2013 21:41:00 GMT+0000 (Coordinated
Universal Time)
The parsed date from input is: Sun Sep 22 2013 20:03:46 GMT+0530 (India
Standard Time)

import java.text.SimpleDateFormat;
import java.util.Date;

public class Test{
public static void main(String[] args) {
String pattern=args[0];
String dtstr = args[1];
SimpleDateFormat sdf = new SimpleDateFormat(pattern);
Date date = new Date();
try {
//
String fmtd = sdf.format(date);
System.out.println("The formatted string: " +fmtd);
System.out.println("The parsed date is: " +
sdf.format(sdf.parse(fmtd)));
System.out.println("The parsed date from input is: " +
sdf.format(sdf.parse(dtstr)));
} catch (Exception e) {
e.printStackTrace();
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #3