Invalid format of timestamp

Elastic Stack Elasticsearch
1

hi i loaded data in elasticsearch ,this is a sample of it

{"index":{"_index":"retails","_type":"information"}}
{"@timestamp":"11-26-2016","product":{"name":"Bread - French Baquette"},"price":"$424.77","customers":3622,"color":"Fuscia","promotion":{"periode":"01-04-2017"}}

i defined then the mapping as below
PUT /retail
{

"mappings" : {
"information" : {
"properties" : {
"@timestamp": {
"type": "date",
"format": "mm-dd-yyyy"
},
"product" : {
"properties":{
"name": {"type" :"text"}
}
},

"customers" : { "type" : "integer" },
"color":{"type": "text"},
"promotion": {
  "properties":{
    "period": { "type": "date"}
  }
}

}
}

}

}

i loaded data with the command
curl -XPOST 'http://localhost:9200/_bulk' --data-binary @retails.json

an error appear
{"took":454,"errors":true,"items":[{"index":{"_index":"retail","_type":"information","_id":"AV1vGJRJ64LfEy1i7EhN","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse [@timestamp]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: "05-17-2017" is malformed at "-2017""}}}}

how can i fix this problem !!!

2

Hi,

Your format is not valid, check here:

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/mapping-date-format.html#mapping-date-format

and here

http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormat.html

Month is MM

3

i changed it into MM-dd-yyy but the same error appears

this is the mapping
PUT /retail
{

"mappings" : {
"information" : {
"properties" : {
"@timestamp": {
"type": "date",
"format": "MM-dd-yyyy"
},
"product" : {
"properties":{
"name": {"type" :"text"}
}
},

"customers" : { "type" : "integer" },
"color":{"type": "text"},
"promotion": {
  "properties":{
    "period": { "type": "date"}
  }
}

}
}

}

}

this is the error
@timestamp]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: "11-26-2016" is malformed at "-26-2016""

4

Note that if you change your mapping, you have to create a new Index with this mapping.

5

i deleted the first one and recreate it

6

Can you provide result of a GET mapping api on your index plz ?

GET /retail/_mapping/information

(https://www.elastic.co/guide/en/elasticsearch/reference/5.5/indices-get-mapping.html)

7

{
"retail": {
"mappings": {
"information": {
"properties": {
"@timestamp": {
"type": "date",
"format": "MM-dd-yyyy"
},
"color": {
"type": "text"
},
"customers": {
"type": "integer"
},
"price": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"product": {
"properties": {
"name": {
"type": "text"
}
}
},
"promotion": {
"properties": {
"period": {
"type": "date"
},
"periode": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
}

this is the result

8

Seems OK, Can you try to load your datas without the @timestamp field to test if ok ?

9

Are you putting in the good index ? "retails" vs "retail" ?

10

yes , i have just verified the index

11

the problem maintains with the timestamp could you give an example of a file containing a timestamp field and how we should map it !!!

12

I have tested on a 2.4 cluster, here are my commands:

retails.json

{  
  "mappings":{  
    "information":{  
	"properties":{  
	  "@timestamp":{  
	      "type":"date",
	      "format":"MM-dd-yyyy"
	  },
	  "color":{  
	      "type":"string"
	  },
	  "customers":{  
	      "type":"integer"
	  },
	  "price":{  
	      "type":"string"
	  },
	  "product":{  
	      "properties":{  
		"name":{  
		    "type":"string"
		}
	      }
	  },
	  "promotion":{  
	      "properties":{  
		"period":{  
		    "type":"date"
		},
		"periode":{  
		    "type":"string"
		}
	      }
	  }
	}
    }
  }
}

Create an index:

curl -XPUT 'http://111.11.11.11:9200/xavier-test-retail' -d @retail.json

retail-data.json

{  "index":{  "_index":"xavier-test-retail", "_type":"information" }}
{  "@timestamp":"11-26-2016", "product":{  "name":"Bread - French Baquette"   },   "price":"$424.77",   "customers":3622,   "color":"Fuscia",   "promotion":{        "periode":"01-04-2017"   }}

Load datas:

curl -XPOST 'http://111.11.11.11:9200/_bulk' --data-binary @retail-data.json

result:

{"took":12,"errors":false,"items":[{"create":{"_index":"xavier-test-retail","_type":"information","_id":"AV1vrlzWBiXCHyywzxDb","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"status":201}}]}

Data is correct in the index.

1 Like
13

i did as you did in the last post but when i went to the management tool in kinbana to create an index pattern i couldn't choose time based event

have a look

kibana.PNG705×242 36.1 KB

but i want to creaea an index pattern with time based event

14

i did find the problem it is about a bracket in the mapping thank you for your help

15

Ok cool! You can close this ticket with the solution button.

Nota: $424.77 is very very expensive for a French Baguette !!! :wink:

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.