Index data conflict with another

Elastic Stack Beats
1

Hi All,

I installed Heartbeat and changed the pipeline to be (Heartbeat ==> Logstash ===> Elasticsearch) and I loaded the heartbeat dashboards,
Also I have its logs, and logs forwarded from (IBM Qradar),
My issue is the document in qradar index is the same for the heartbeat index, and when I stop the heartbeat I get the correct document.
(Heartbeat document overwrite other logs )
I tested it on version 7.6.2
Also tested it on 7.9.2 but the logs shipped using filebeat overwrite the heartbeat documents

2

It's not really clear what the issue is here sorry.

Can you explain a little more on your setup, what you use Filebeat and Heartbeat for, and how they send to Elasticsearch

3

I am using Filebeat to collect our applications logs,
Heartbeat is used to monitor the application (soap-http-tcp), and changed the pipeline of heartbeat to send the response to logstash to parse some data, this step requires loading heartbeat dashboards in kibana,
All logs sent to logstassh then to elasticsearch.

4

Does Filebeat send to Logstash?

5

Yes sure for parsing

6

Ok, so is the Filebeat and Heartbeat ending up in the same index? Is that what you mean by conflicting?

7

No sir,
I have pipeline for each one (one for qradar as syslog and one for filebeat and one for Heartbeat) and each one is created and I created index pattern for each one of them, and to make sure that I am working right I tried to stop the heartbeat service and everything goes okay, each index has its own document

8

So what's the actual issue then?

9

When I open filebeat index or syslog index it gives me the documents of heartbeat, when I stop heartbeat the documents match's it's index

10

Then you may have overlap in your Logstash config.

11

Each pipeline has its own port and input and no filters
Please advice how its overlapped

12

Does each output have it's own unique index name?

13

Sure

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.