Python doesn't send docs to elasticsearch after set mappings


(Filippo) #1

I have uploaded many time doc to elasticseatch without specify the mapping without any problem, instead this time I need to map the fields in order to set the format of my timestamp field.

When I launch the script it seems to run without problems but at the end I can't find any documents uploaded at the index in elasticsearch.

Here my script

from elasticsearch import Elasticsearch

import json

es= Elasticsearch()

INDEX_NAME = 'technogym_error_time'
TYPE_NAME = 'technogym_error_time'
ID_FIELD = 'id'

bulk_data = []

with open("C:/Users/Filippo-Leoncini/Documents/Lavoro/IBM/Technogym/Dashboard/dati/dati_partenza/dati_nuovi/20170515 11.42RetrainExtraction") as data_file:
data =json.load(data_file)

for i in data["rows"]:
document_id = i["id"]
time_stamp = i["doc"]["timestamp"][5:-4]
error = i["doc"]['review_reason']
node_list=({"timestamp": time_stamp.encode('utf-8'),"document_id": document_id.encode('utf-8'),"error":error.encode('utf-8')})
op_dict = {
"index": {
"_index": INDEX_NAME,
"_type": TYPE_NAME,
"_id": document_id
}
}
bulk_data.append(op_dict)
bulk_data.append(node_list)

settings = {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"technogym_error_time": {
"properties": {
"timestamp":{
"type":"date",
"format":"dd MM YYYY HH:mm:ss"
},
"error":{
"type":"string"
}
}
}
}
}

es.indices.create(index= INDEX_NAME, body=settings)
es.bulk(body= bulk_data, index = INDEX_NAME, doc_type = TYPE_NAME)

Here a screenshot of my elasticsearch indices

And here is what I get on Kibana

Thanks


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.