How to create new field into an elasticsearch index using logstash and python

Elastic Stack Logstash
1

Hi,
im trying to generate tweets sentiments with a python script, and want to create a new field into elasticsearch, that contains the sentiment (ss["compound"] value) for each tweet.
Any idea???
Thank you

t1
t1.PNG715×271 19.8 KB

2

So you want to update existing documents with a sentiment field?

If yes, any particular reason you want to involve Logstash in this?

3

Hi Magnusbaeck,

So is it possible to edit the ES index directly without logstash by populating it with new field?

best
Mudireddy

4

Yes, just use the Python API's index call. It will by default create a new document with the specified id or, if the document already exists, update it with the new contents you supply.

5

Logstash filters some words that I dont want to have into my document.But the second problem is that i should write this python script in the logstash filter. I think i should use ruby filter.

6

I dont understand. Can you give me an example please .

7

Something like this:

doc = hit["_source"]
doc["sentiment"] = sid.polarity_scores(x)
es.index(index="t1", doc_type=hit["_type"], id=hit["_id"], body=doc)

Reference documentation is at https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.index
and there's at least one other example at https://github.com/elastic/elasticsearch-py.

This group is for Logstash questions so usage of elasticsearch-py doesn't fit in.

1 Like
8

Okey.
Thanks for help

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.