# Fluentd performance is degraded **URL:** \ **Category:** Elasticsearch\ **Created:** [July 30, 2021, 9:15am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060 "2021-07-30T09:15:25Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Voula\_Mikr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/voula_mikr/32/58451_2.png) [@Voula\_Mikr](https://discuss.elastic.co/u/Voula_Mikr)\ **Post date:** [July 30, 2021, 9:15am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060/1 "2021-07-30T09:15:25Z")
In a deployment with elasticsearch 8.x, fluentd pod performance has been degraded and following log is printed whenever a new message is received and it is about to be delivered in elastic. warning: 299 Elasticsearch-7.8.0-8e340a0952a52f4feb290854ae1051e9a83ca9de "[types removal] Specifying types in bulk requests is deprecated." Is this log relevant to the loss of the performance? What should I check in order to increase performance? Thank you in advance. ---
**Author:** ![John\_Xanthopoulos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_xanthopoulos/32/60540_2.png) [@John\_Xanthopoulos](https://discuss.elastic.co/u/John_Xanthopoulos)\ **Post date:** [July 30, 2021, 10:33am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060/2 "2021-07-30T10:33:01Z")
Hi, we also see these kind of errors.. 2021-07-30 10:12:16 +0000 [warn]: #1 dump an error event: error\_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch" location=nil tag="pmdata" time=2021-07-30 10:12:14.688068202 +0000 record={"vnf\_name"=\>"UDM", "vnf\_type"=\>"UDM", "distinguished\_name"=\>"NTF3", "counter\_name"=\>"-counter-name-15", "counter\_group\_name"=\>"counterGroupUDM1", "measurement\_time"=\>"2021-07-23T10:12:14.687369Z", "counter\_value"=\>3, "counter\_display\_name"=\>"UDM-NTF3-counterGroupUDM1-counter-name-2", "is\_dynamic"=\>"True", "date"=\>"2021-07-23", "\_hash"=\>"MjUyMzRhNDctOWFjMC00YTQ0LTlkZTktNGI4YjAyNzQ3M2Vj"} ---
**Author:** ![abrx](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abrx/32/43881_2.png) [@abrx](https://discuss.elastic.co/u/abrx)\ **Post date:** [July 30, 2021, 12:49pm UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060/3 "2021-07-30T12:49:13Z")
Hi, @Voula_Mikr The type in indices, that allowed to group different kind of documents in the same index, is deprecated since Elasticsearch 6.x and removed with 7.x (see [here](https://dzone.com/articles/the-removal-of-mapping-types-in-elasticsearch-6-th) for details, but in short the idea is to keep only the data that look similar in same indices, and to use custom fields to distinguish them if needed). So you should remove types occurence in fluend configuration (as it's now useless), but I don't think it could cause performance issues, unless you try to index/query huge amount of data. However you can check: - if there's any pending tasks / relocating shards on your cluster with `_cluster/health` endpoint - if there is rejected messages (on Elasticsearch logs) that can make your fluentd to buffer unsent data and fill its memory (and more generally if there's any resource saturation on both side) @John_Xanthopoulos I think that your issue is different, as the data is rejected by Elasticsearch and no mention of type appears on your logs. You may consider looking: - if your cluster is healthy and no proxy prevent you from writing in it - if there's some mismatch on data types you try to index vs your mapping (for instance, string vs date or so) - if your index is read-only due to a previous saturation, via the `/_settings` endpoint, there should be no field like `index.blocks.read_only_allow_delete` (if there is, you can PUT a null value to it to remove the constraint) ---
**Author:** ![John\_Xanthopoulos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_xanthopoulos/32/60540_2.png) [@John\_Xanthopoulos](https://discuss.elastic.co/u/John_Xanthopoulos)\ **Post date:** [July 31, 2021, 6:05am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060/4 "2021-07-31T06:05:31Z")
> [@abrx](#): > > \_cluster/health Thanks abrx, with regards the log I posted the issue was under the category --\> \* if there's some mismatch on data types you try to index vs your mapping (for instance, string vs date or so) With regards this log --\> warning: 299 Elasticsearch-7.8.0-8e340a0952a52f4feb290854ae1051e9a83ca9de "[types removal] Specifying types in bulk requests is deprecated. We managed to suppress it by configuring fluentd with "suppress\_type\_name true" and log was vanished. Performance issue is still there though, output of cluster health is ''' { "active\_primary\_shards": 53, "active\_shards": 106, "active\_shards\_percent\_as\_number": 100.0, "cluster\_name": "nc1296-admin-ns-zts-lms", "delayed\_unassigned\_shards": 0, "initializing\_shards": 0, "number\_of\_data\_nodes": 2, "number\_of\_in\_flight\_fetch": 0, "number\_of\_nodes": 8, "number\_of\_pending\_tasks": 0, "relocating\_shards": 0, "status": "green", "task\_max\_waiting\_in\_queue\_millis": 0, "timed\_out": false, "unassigned\_shards": 0 } ''' So it looks that there is no issue there right? Additionally I dont see any rejected messages neither in fluentd nor in elasticsearch. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 28, 2021, 6:05am UTC](https://discuss.elastic.co/t/fluentd-performance-is-degraded/280060/5 "2021-08-28T06:05:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.