# How to create new field into an elasticsearch index using logstash and python

**URL:** <https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546>\
**Category:** Logstash\
**Created:** [February 26, 2018, 6:27pm UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546 "2018-02-26T18:27:45Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![soukaina](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@soukaina](https://discuss.elastic.co/u/soukaina)\
**Post date:** [February 26, 2018, 6:27pm UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/1 "2018-02-26T18:27:45Z")

</div>

Hi,  
im trying to generate tweets sentiments with a python script, and want to create a new field into elasticsearch, that contains the sentiment (ss["compound"] value) for each tweet.  
Any idea???  
Thank you

 ![t1](https://us1.discourse-cdn.com/elastic/original/3X/9/a/9a023f2a04676414ade693772d8f6f834d45988c.PNG)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 27, 2018, 7:03am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/2 "2018-02-27T07:03:59Z")

</div>

So you want to update existing documents with a sentiment field?

If yes, any particular reason you want to involve Logstash in this?

---

<div class="post-metadata">

**Author:** ![Srikanth\_FT](https://avatars.discourse-cdn.com/v4/letter/s/d9b06d/32.png) [@Srikanth\_FT](https://discuss.elastic.co/u/Srikanth_FT)\
**Post date:** [February 27, 2018, 8:41am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/3 "2018-02-27T08:41:13Z")

</div>

Hi Magnusbaeck,

So is it possible to edit the ES index directly without logstash by populating it with new field?

best  
Mudireddy

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 27, 2018, 8:55am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/4 "2018-02-27T08:55:15Z")

</div>

Yes, just use the Python API's index call. It will by default create a new document with the specified id or, if the document already exists, update it with the new contents you supply.

---

<div class="post-metadata">

**Author:** ![soukaina](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@soukaina](https://discuss.elastic.co/u/soukaina)\
**Post date:** [February 27, 2018, 9:27am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/5 "2018-02-27T09:27:27Z")

</div>

Logstash filters some words that I dont want to have into my document.But the second problem is that i should write this python script in the logstash filter. I think i should use ruby filter.

---

<div class="post-metadata">

**Author:** ![soukaina](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@soukaina](https://discuss.elastic.co/u/soukaina)\
**Post date:** [February 27, 2018, 9:32am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/6 "2018-02-27T09:32:35Z")

</div>

I dont understand. Can you give me an example please .

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 27, 2018, 9:39am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/7 "2018-02-27T09:39:56Z")

</div>

Something like this:

```python
doc = hit["_source"]
doc["sentiment"] = sid.polarity_scores(x)
es.index(index="t1", doc_type=hit["_type"], id=hit["_id"], body=doc)

```

Reference documentation is at [https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.index](https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.index)  
and there's at least one other example at [https://github.com/elastic/elasticsearch-py](https://github.com/elastic/elasticsearch-py).

This group is for Logstash questions so usage of elasticsearch-py doesn't fit in.

---

<div class="post-metadata">

**Author:** ![soukaina](https://avatars.discourse-cdn.com/v4/letter/s/4da419/32.png) [@soukaina](https://discuss.elastic.co/u/soukaina)\
**Post date:** [February 27, 2018, 10:36am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/8 "2018-02-27T10:36:36Z")

</div>

Okey.  
Thanks for help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 27, 2018, 10:36am UTC](https://discuss.elastic.co/t/how-to-create-new-field-into-an-elasticsearch-index-using-logstash-and-python/121546/9 "2018-03-27T10:36:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.