How to create new field into an elasticsearch index using logstash and python

Hi,
im trying to generate tweets sentiments with a python script, and want to create a new field into elasticsearch, that contains the sentiment (ss["compound"] value) for each tweet.
Any idea???
Thank you

So you want to update existing documents with a sentiment field?

If yes, any particular reason you want to involve Logstash in this?

Hi Magnusbaeck,

So is it possible to edit the ES index directly without logstash by populating it with new field?

best
Mudireddy

Yes, just use the Python API's index call. It will by default create a new document with the specified id or, if the document already exists, update it with the new contents you supply.

Logstash filters some words that I dont want to have into my document.But the second problem is that i should write this python script in the logstash filter. I think i should use ruby filter.

I dont understand. Can you give me an example please .

Something like this:

doc = hit["_source"]
doc["sentiment"] = sid.polarity_scores(x)
es.index(index="t1", doc_type=hit["_type"], id=hit["_id"], body=doc)

Reference documentation is at https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.index
and there's at least one other example at https://github.com/elastic/elasticsearch-py.

This group is for Logstash questions so usage of elasticsearch-py doesn't fit in.

1 Like

Okey.
Thanks for help

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.