Logstash is importing more data than supposed

Elastic Stack Logstash
1

Hello guys,

I'm currently importing data from csv through logstash to elasticsearch.

I'm importing 26595 lines but logstash seems to be importing or more than 26595 or less lines.

Is there any explanation for this?

Thank you very much,

2

Without further details it's impossible to help. So are lines being duplicated? Which ones? How many lines do you end up with in ES? Was the index really empty when you started the import?

I'm importing 26595 lines but logstash seems to be importing or more than 26595 or less lines.

I can't parse this sentence.

3

Hi magnusbaeck,

Thanks for the answer.

It is difficult to answer all that question because I can't figure it out the lines that are being duplicated because there are many duplicated lines in the excel. So in elasticsearch I can't really distinguish.

Relatively to your second question. It depends, I have done some tests and sometimes puts more 10 lines than supposed, sometimes more 2 lines.

Yes, I ensure that the index was empty before importing.

You don't have any problem when importing this much data to elasticsearch from logstash?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.