I want to save documents in an Index by timestamp.
I am saving documents every second, so I want to see a document being created every second.
But when I save the document by passing new timestamp value, a new document is not created but the same existing document is updated.
So instead of seeing documents being saved per second, I see the same document getting update every second.
Below is the mapping information about my index.
{
"iotdevice": {
"aliases": {},
"mappings": {
"sensordata": {
"properties": {
"nodeID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sensorData": {
"type": "float"
},
"sensorID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sensorName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"sensorState": {
"type": "long"
},
"sensorType": {
"type": "long"
},
"siteIP": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timestamp": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
},
"settings": {
"index": {
"creation_date": "1508352509436",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "ZxcszX-LTzatqYCQV3z4QA",
"version": {
"created": "5030299"
},
"provided_name": "iotdevice"
}
}
}
}
I am saving the document every second by using the bulk insert operation.
But after saving 100's of documents per second, I am seeing only 1 record with the most recent timestamp/
https://search-sensordata-hima7jdvzx6rkn6tpprlqlyp4e.us-east-1.es.amazonaws.com/iotdevice/_search/?size=1000&pretty=1
Response ->
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "iotdevice",
"_type": "sensordata",
"_id": "ID002",
"_score": 1,
"_source": {
"sensorID": "ID002",
"sensorName": "Sensor_SFO",
"sensorState": 1,
"timestamp": "2017-10-19 16:16:44"
}
}
}
]
}
}