# Index data conflict with another

**URL:** https://discuss.elastic.co/t/index-data-conflict-with-another/254271
**Category:** Beats
**Tags:** heartbeat
**Created:** [November 4, 2020, 1:01pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271 "2020-11-04T13:01:02Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 4, 2020, 1:01pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/1 "2020-11-04T13:01:03Z")

</div>

Hi All,

I installed Heartbeat and changed the pipeline to be (Heartbeat ==\> Logstash ===\> Elasticsearch) and I loaded the heartbeat dashboards,  
Also I have its logs, and logs forwarded from (IBM Qradar),  
My issue is the document in qradar index is the same for the heartbeat index, and when I stop the heartbeat I get the correct document.  
(Heartbeat document overwrite other logs )  
I tested it on version 7.6.2  
Also tested it on 7.9.2 but the logs shipped using filebeat overwrite the heartbeat documents

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 3:31am UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/2 "2020-11-05T03:31:18Z")

</div>

It's not really clear what the issue is here sorry.

Can you explain a little more on your setup, what you use Filebeat and Heartbeat for, and how they send to Elasticsearch

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 8:51am UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/3 "2020-11-05T08:51:55Z")

</div>

I am using Filebeat to collect our applications logs,  
Heartbeat is used to monitor the application (soap-http-tcp), and changed the pipeline of heartbeat to send the response to logstash to parse some data, this step requires loading heartbeat dashboards in kibana,  
All logs sent to logstassh then to elasticsearch.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 9:42pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/4 "2020-11-05T21:42:31Z")

</div>

Does Filebeat send to Logstash?

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 9:56pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/5 "2020-11-05T21:56:10Z")

</div>

Yes sure for parsing

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 9:57pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/6 "2020-11-05T21:57:47Z")

</div>

Ok, so is the Filebeat and Heartbeat ending up in the same index? Is that what you mean by conflicting?

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 10:14pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/7 "2020-11-05T22:14:56Z")

</div>

No sir,  
I have pipeline for each one (one for qradar as syslog and one for filebeat and one for Heartbeat) and each one is created and I created index pattern for each one of them, and to make sure that I am working right I tried to stop the heartbeat service and everything goes okay, each index has its own document

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 10:16pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/8 "2020-11-05T22:16:45Z")

</div>

So what's the actual issue then?

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 10:36pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/9 "2020-11-05T22:36:17Z")

</div>

When I open filebeat index or syslog index it gives me the documents of heartbeat, when I stop heartbeat the documents match's it's index

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 10:37pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/10 "2020-11-05T22:37:58Z")

</div>

Then you may have overlap in your Logstash config.

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 11:15pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/11 "2020-11-05T23:15:09Z")

</div>

Each pipeline has its own port and input and no filters  
Please advice how its overlapped

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 5, 2020, 11:36pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/12 "2020-11-05T23:36:03Z")

</div>

Does each output have it's own unique index name?

---

<div class="post-metadata">

### Author: ![mustafa.husny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mustafa.husny/32/77154_2.png) [@mustafa.husny](https://discuss.elastic.co/u/mustafa.husny)
#### Post date: [November 5, 2020, 11:55pm UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/13 "2020-11-05T23:55:18Z")

</div>

Sure

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 4, 2020, 1:55am UTC](https://discuss.elastic.co/t/index-data-conflict-with-another/254271/14 "2020-12-04T01:55:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
