How to create new field into an elasticsearch index using logstash and python


im trying to generate tweets sentiments with a python script, and want to create a new field into elasticsearch, that contains the sentiment (ss["compound"] value) for each tweet.
Any idea???
Thank you

(Magnus Bäck) #2

So you want to update existing documents with a sentiment field?

If yes, any particular reason you want to involve Logstash in this?

(R) #3

Hi Magnusbaeck,

So is it possible to edit the ES index directly without logstash by populating it with new field?


(Magnus Bäck) #4

Yes, just use the Python API's index call. It will by default create a new document with the specified id or, if the document already exists, update it with the new contents you supply.


Logstash filters some words that I dont want to have into my document.But the second problem is that i should write this python script in the logstash filter. I think i should use ruby filter.


I dont understand. Can you give me an example please .

(Magnus Bäck) #7

Something like this:

doc = hit["_source"]
doc["sentiment"] = sid.polarity_scores(x)
es.index(index="t1", doc_type=hit["_type"], id=hit["_id"], body=doc)

Reference documentation is at
and there's at least one other example at

This group is for Logstash questions so usage of elasticsearch-py doesn't fit in.


Thanks for help

(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.