# Filebeat hogged the IO

**URL:** <https://discuss.elastic.co/t/filebeat-hogged-the-io/314581>\
**Category:** Beats\
**Tags:** filebeat, libbeat\
**Created:** [September 16, 2022, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581 "2022-09-16T14:35:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![silence-linhl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/silence-linhl/32/110844_2.png) [@silence-linhl](https://discuss.elastic.co/u/silence-linhl)\
**Post date:** [September 16, 2022, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581/1 "2022-09-16T14:35:20Z")

</div>

filebeat hogged the IO

After starting filebeat, I found that the IO of the machine became very high. After the digging, it was found that filebeat kept writing to disk a file named checkpoint.new after it was started.

So I used "checkpoint.new" to search in source code to find which function call produce the file named checkpoint.new. Finally, I found that the checkpoint.new was produced by "WriteCheckpoint" in diskstore.go:292. Before calling "WriteCheckpoint", another function "mustCheckpoint"(store.go:213) was executed to decide wether to call "WriteCheckpoint". The detail code of mustCheckpoint is shown as follows:

```auto

// mustCheckpoint returns true if the store is required to execute a checkpoint

// operation, either by predicate or by some internal state detecting a problem

// with the log file.

func (s *diskstore) mustCheckpoint() bool {

return s.logInvalid || s.checkpointPred(s.logFileSize)

}

```

If s.logInvalid is true, mustCheckpoint always return true. So the store is always required to execute a checkpoint.

I found a warning message with "Incomplete or corrupted log file in" in the log of filebeat. This message will be printed indicating that there was an error loading the log file(store.go:130). At this time, the value of s.logInvalid is equal to err!=nil(store.go:133). Unfortunately in this case err!=nil equals true, i.e. s.logInvali equals true.

As mentioned above, once s.logInvali equals true, the store is always required to execute a checkpoint.

So once there is a problem with the log file, it will lead to frequent checkpoints and finally cause high IO.

Maybe it's a bug ?

diskstore.go: [beats/diskstore.go at main · elastic/beats · GitHub](https://github.com/elastic/beats/blob/main/libbeat/statestore/backend/memlog/diskstore.go)

store.go: [beats/store.go at v7.17.2 · elastic/beats · GitHub](https://github.com/elastic/beats/blob/v7.17.2/libbeat/statestore/backend/memlog/store.go)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2022, 4:35pm UTC](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581/2 "2022-10-14T16:35:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.