# Strange Fluent-Bit logs

**URL:** https://discuss.elastic.co/t/strange-fluent-bit-logs/167881
**Category:** Elasticsearch
**Created:** [February 11, 2019, 3:09pm UTC](https://discuss.elastic.co/t/strange-fluent-bit-logs/167881 "2019-02-11T15:09:58Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![JDev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jdev/32/40795_2.png) [@JDev](https://discuss.elastic.co/u/JDev)
#### Post date: [February 11, 2019, 3:09pm UTC](https://discuss.elastic.co/t/strange-fluent-bit-logs/167881/1 "2019-02-11T15:09:58Z")

</div>

As a log collector, I installed fluent-bit.

It is installed on each node of the cluster. All right. He sends logs to elastic. I see them. All OK!

But if i look in the log of fluent-bit pod, then I see the following. Is this normal, or do I need to configure or configure something?

```
{"took":14959,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.01.17","_type":"flb_type","_id":"U_cD3WgB3lQtmGeDeT7j","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":626,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.01.17","_type":"flb_type","_id":"VPcD3WgB3lQtmGeDeT7j","status":429,"error":{"type":"es_rejected_execution_exception","reason":"rejected execution of processing of [14672][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[kubernetes_cluster-2019.01.17][4]] containing [24] requests, target allocation id: szv_pPlJRECoocKHvAa_Ng, primary term: 2 on EsThreadPoolExecutor[name = elasticsearch-master-2/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@15dfd52a[Running, pool size = 1, active threads = 1, queued tasks = 200, completed tasks = 2849]]"}}},{"index":{"_index":"kubernetes_cluster-2019.0

[2019/02/11 14:43:54] [error] [out_es] could not pack/validate JSON response

{"took":15028,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VNUD3WgBIB2PCaLHefD4","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":12550,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VdUD3WgBIB2PCaLHefD4","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":12900,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VtUD3WgBIB2PCaLHefD4","status":429,"error":{"type":"es_rejected_execution_exception","reason":"rejected execution of processing of [14675][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[kubernetes_cluster-2019.02.08][2]] containing [100] requests, target allocation id: lOBDNeN1QeuVZ1JGnkPxZQ, primary term: 2 on EsThreadPoolExecutor[name = elasticsearc

[2019/02/11 14:43:54] [warn] [out_es] Elasticsearch error

{"took":15028,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VNUD3WgBIB2PCaLHefD4","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":12550,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VdUD3WgBIB2PCaLHefD4","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":12900,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"VtUD3WgBIB2PCaLHefD4","status":429,"error":{"type":"es_rejected_execution_exception","reason":"rejected execution of processing of [14675][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[kubernetes_cluster-2019.02.08][2]] containing [100] requests, target allocation id: lOBDNeN1QeuVZ1JGnkPxZQ, primary term: 2 on EsThreadPoolExecutor[name = elasticsearch-master-2/write,

[2019/02/11 14:43:57] [error] [out_es] could not pack/validate JSON response

{"took":15015,"errors":true,"items":[{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"4NUD3WgBIB2PCaLHhfZj","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"_seq_no":12704,"_primary_term":2,"status":201}},{"index":{"_index":"kubernetes_cluster-2019.02.08","_type":"flb_type","_id":"4dUD3WgBIB2PCaLHhfZj","status":429,"error":{"type":"es_rejected_execution_exception","reason":"rejected execution of processing of [14802][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[kubernetes_cluster-2019.02.08][4]] containing [65] requests, target allocation id: VS1lozI6RVaC-nBnF5T3yw, primary term: 2 on EsThreadPoolExecutor[name = elasticsearch-master-2/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@15dfd52a[Running, pool size = 1, active threads = 1, queued tasks = 200, completed tasks = 2898]]"}}},{"index":{"_index":"kuber

[2019/02/11 14:43:57] [warn] [out_es] Elasticsearch error

```

Thsnks

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 11, 2019, 11:24pm UTC](https://discuss.elastic.co/t/strange-fluent-bit-logs/167881/2 "2019-02-11T23:24:33Z")

</div>

It looks like you are overloading your cluster, as the bulk threadpool is getting full and rejecting requests.

Do you have Monitoring enable?

---

<div class="post-metadata">

### Author: ![JDev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jdev/32/40795_2.png) [@JDev](https://discuss.elastic.co/u/JDev)
#### Post date: [February 12, 2019, 1:03pm UTC](https://discuss.elastic.co/t/strange-fluent-bit-logs/167881/3 "2019-02-12T13:03:38Z")

</div>

Yes, I already googled it. But the fact is that I just installed it and it happens right at the start. I have a small cluster.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 12, 2019, 3:03pm UTC](https://discuss.elastic.co/t/strange-fluent-bit-logs/167881/4 "2019-03-12T15:03:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
