The error about data type filed

Elastic Stack Logstash
1

hi everyone,
I foud the weird problem about my es mapping,it's as flows:
curl -XPUT 'localhost:9200/v_ajusted_fund_v1' -d '{
"mappings" : {
"v_ajusted_fund" : {
"properties" : {
"period" : {"type" : "integer"},
"project_code" : {"type" : "string","index" : "not_analyzed"},
"labor_budget": {"type" : "double"},
"year_labor_budget": {"type" : "double"},
"total_p_project_fund": {"type" : "double"},
"year_fund": {"type" : "double"},
"total_year_fund": {"type" : "double"},
"total_f_project_fund": {"type" : "double"},
"create_time": {"type" : "date"}
}
}
}
}'

the table field create_time is date type and the value is:"2013-09-28" .when I excute logstash order,is has an error in console:

ParsingException[failed to parse [create_time]]; nested: IllegalArgumentException[Invalid format: "2014-09-28 00:00:00 +0800" is malformed at " 00:00:00 +0800"];
at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:343)
at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:318)
at org.elasticsearch.index.mapper.DocumentParser.parseAndMergeUpdate(DocumentParser.java:765)
at org.elasticsearch.index.mapper.DocumentParser.parseDynamicValue(DocumentParser.java:652)
at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:451)
at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:271)
at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:131)
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:79)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304)
at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:500)
at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.java:481)
at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:214)
at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:595)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:263)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260)
at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid format: "2014-09-28 00:00:00 +0800" is malformed at " 00:00:00 +0800"
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType.parseStringValue(DateFieldMapper.java:362)
at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:528)
at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:335)

anyone who I has an solution? very appreciate!

Levi

2

This seems more like an ES question than a Logstash one, but I think the problem is quite clear: the create_time field is mapped as "date", which without further configuration requires a strict ISO8601 timestamp with or without a time but the timestamp you're trying to cram into the field doesn't conform to that definition. Either remove the time or map the create_time field with a time format that matches your input.

3

hi magnus,
I try using the format just like as bellow:
"create_time": {"type" : "date","format":"yyyy-MM-dd"},But I found it still has the same error! thanks!

4

BTW,I learn from this url:
https://www.elastic.co/guide/en/elasticsearch/reference/2.3/date.html#multiple-date-formats

I also try using this:
"create_time": {"type" : "date","format":"yyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}
but it's not work!