From the time to time Elastic's docs.count value is updated by logstash. Is it normal?

[german (Germán Hernández)]

Hi everybody,

I have a little question about elastic's docs.count as I have noticed that it's not updated constantly. For example:

(Don't pay attention to credentials. It's only a lab test).

The 3487 docs.count value hasn't updated since ten minutes ago, although Logstash continues processing:

On top of that, the docs.count quantity it doesn't make sense if I compare with the number of lines which Logstash has processed from the logs. Having said that, is a document the same that a row of a log?

Thanks for your time.

[Christian_Dahlqvist]

In older versions of Elasticsearch the refresh interval, which determines how frequently data is written to segments and made available for search, was followed stringently and the lag between documents being ingested and showing up was predictable. As this is an expensive operation it was often recommended to increase this interval in order to improve ingest throughput. In recent versions I believe the interval is still used, but refreshes may be postponed if there are no searches against the indices. If you run a search, this should trigger a refresh and the numbers get updated. The fact that the document count is not getting updated does not necessarily mean there is a problem.

[german (Germán Hernández)]

Thanks for your answer Cristian.

I got it. I just test it, running "_search" using curl and the counter was refreshed. Regarding the documents, is the same a document and a row from a text file? I mean, if the file text contains 1000 rows and all of them match within logstash filter, I'm going to get 1000 Elastic's documents?

[Christian_Dahlqvist]

If you do not do any multiline processing or splitting that is generally the case.

[german (Germán Hernández)]

Thanks for your time Christian.