Overwriting new document?

Elastic Stack Elasticsearch
1

Hi,

Recently I did a fresh install of elk 7.7 . I did a successfull attempt of creaing a POC and now try to build up a full environment. However, When I send a log via filebeatand logstash to ES, somehow every new document overwrites the previous one. What am I doing wrong?
Have not seen this in my POC now on my previous production environment of ELK 6.7

Any suggestions where to look?

2

That is often caused by you setting the document_id in the Elasticsearch output in Logstash and that the field you are using is wrong or not defined.

3

Thnx for the quick reponse. WIll have a look at that, however, I am surprised that that was not the case than with te POC install, which used the same output definitions.

4

Maybe the input has changed and the field is no longer present? Have a look at the ID of the document being indexed. That should show if it is this causing it or not.

5

It really DOES help if one puts the appropriate filter conf file in the conf.d as well :frowning: :wink:

Thnx for pointing me in te right direction.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.