# New documents are not getting added to an already existing index
**URL:** https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026
**Category:** Elasticsearch
**Created:** [July 7, 2021, 6:23am UTC](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026 "2021-07-07T06:23:51Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Krishna\_Sathya\_A\_C](https://discuss.elastic.co/u/Krishna_Sathya_A_C)
#### Post date: [July 7, 2021, 6:23am UTC](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026/1 "2021-07-07T06:23:51Z")
Hi,
I am new to the ELK stack workflow, currently I am doing a project that pulls live feed from twitter every time a user requests and I want to display the results using Kibana. In my project instead of using Logstash to collect data I have used tweepy library in python, and then connected it to Elastic Search where I created an index to store my incoming data.
**The problem:**
The code is working but my problem is that the incoming doccuments are getting stored only if I give a new index name every other time my new set of live feeds come in. If I wanted to use the same index name to add all incoming documents to the existing ones the new documents are not updating.
I have attached my code snipped below:
```auto
def filterKeys(document):
return {key: document[key] for key in use_these_keys }
from elasticsearch import Elasticsearch
from elasticsearch import helpers
es_client = Elasticsearch(http_compress=True)
def doc_generator(df):
df_iter = df.iterrows()
#index, document = next(df_iter)
for index, document in df_iter:
yield {
"_index": 'twitter_req',
"_type": "twitter_twp",
"_id" : f"{document['ID']}",
"_source": filterKeys(document),
}
#raise StopIteration
helpers.bulk(es_client, doc_generator(twitter_feeds))
```
Can any of you kindly tell me where I am going wrong and how to make it so that incoming documents get automatically updated every time the code runs.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 7, 2021, 11:37pm UTC](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026/2 "2021-07-07T23:37:38Z")
Given that you have said you are not using logstash I do not understand why you would post this in the logstash forum. Perhaps you should edit it to move it to the elasticsearch forum.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 4, 2021, 11:38pm UTC](https://discuss.elastic.co/t/new-documents-are-not-getting-added-to-an-already-existing-index/278026/3 "2021-08-04T23:38:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.