# Adding data to existing index for which dashboards are created

**URL:** https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700
**Category:** Logstash
**Created:** [January 17, 2019, 8:41pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700 "2019-01-17T20:41:06Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![samnayak](https://avatars.discourse-cdn.com/v4/letter/s/919ad9/32.png) [@samnayak](https://discuss.elastic.co/u/samnayak)
#### Post date: [January 17, 2019, 8:41pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/1 "2019-01-17T20:41:06Z")

</div>

Hi All,

I am trying to add new data to an existing index in Elasticsearch by running the same config file in Logstash. I already have kibana dashboard created for that index.

My doubt is, does Logstah copy again the old data with new data? Does it affect the created dashboards?

Thanks!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 17, 2019, 9:16pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/2 "2019-01-17T21:16:57Z")

</div>

If you process the same event through an elasticsearch output a second time then unless you are setting the document\_id option you will get a second, duplicate, document in the index. It depends on what your input data looks like, but you may be able to generate a unique document\_id using a fingerprint filter.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 17, 2019, 9:20pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/3 "2019-01-17T21:20:22Z")

</div>

If Logstash is pointing to the same location for the new data, then it should ignore any files it has processed thanks to its `sincedb`.

And yes it will effect the dashboards, because those will be showing the additional data.

---

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [January 17, 2019, 9:50pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/4 "2019-01-17T21:50:05Z")

</div>

yes you can do it. You have to make sure you have same document\_id.  
everything is very much depend on uniq document\_id.  
When I first starting working on this I had to go through this. it is not explain properly anywhere.

Anyway what ever you do make sure you create your own document\_id from your data so you can update.

on your output section you have to use  
action=\>"update" and it will update document if it exist

for example,  
you already have three field from somewhere. job, machine\_name,center in your index "job\_data"

input {  
read data -------- one of the field is job# which is uniq  
and it also reads machine\_name, center  
}  
output {  
document\_id =\> %{job}  
}

now you said are going to read more field to add that in to record  
you can read job, machine\_name,center. and new field "status"

use  
action =\> "update" on output section and it will update record and add new field for you.

it is always good to post some example that way whoever is trying to give you answer knows what you trying to achieve.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 17, 2019, 9:55pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/5 "2019-01-17T21:55:15Z")

</div>

> [@elasticforme](#):
>
> everything is very much depend on uniq document\_id.  
> When I first starting working on this I had to go through this. it is not explain properly anywhere.

It's not though, you don't **need** to define your document ID.

---

<div class="post-metadata">

### Author: ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)
#### Post date: [January 17, 2019, 9:58pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/6 "2019-01-17T21:58:11Z")

</div>

yes you don't have to define. then it will generate automatic document\_id

and now when you try to update how does it do it?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 14, 2019, 10:11pm UTC](https://discuss.elastic.co/t/adding-data-to-existing-index-for-which-dashboards-are-created/164700/7 "2019-02-14T22:11:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
