# Filebeat v7.14.0 Error in Loading Pipeline

**URL:** <https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 6, 2021, 6:35am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407 "2021-09-06T06:35:27Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mary\_Cane\_Bandohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mary_cane_bandohan/32/81420_2.png) [@Mary\_Cane\_Bandohan](https://discuss.elastic.co/u/Mary_Cane_Bandohan)\
**Post date:** [September 6, 2021, 6:35am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/1 "2021-09-06T06:35:27Z")

</div>

Can someone please help us troubleshoot the error we're having after installing Filebeat version 7.14.0 in our CentOS server?  
We have disabled ILM in the filebeat.yml file.  
Our Elasticsearch and Kibana versions are currently both version 7.8.1.  
According to the [Support Matrix](https://www.elastic.co/fr/support/matrix#matrix_compatibility), they are compatible.  
Is there anything else we could check?  
Log errors:

```auto
2021-09-06T05:33:00.539+0400 ERROR [modules] fileset/factory.go:163 Error loading pipeline: 1 error: error loading pipeline for fileset nginx/access: 
{"error":{"root_cause":[{"type":"exception","reason":"org.elasticsearch.common.xcontent.XContentParseException: [1:2] [script] unknown field [description]","processor_type":"script"}],"type":"exception","reason":"org.elasticsearch.common.xcontent.XContentParseException: [1:2] [script] unknown field [description]","processor_type":"script","caused_by":{"type":"x_content_parse_exception","reason":"[1:2] [script] unknown field [description]"}},"status":500}. Response body: {"error":{"root_cause":[{"type":"exception","reason":"org.elasticsearch.common.xcontent.XContentParseException: [1:2] [script] unknown field [description]","processor_type":"script"}],"type":"exception","reason":"org.elasticsearch.common.xcontent.XContentParseException: [1:2] [script] unknown field [description]","processor_type":"script","caused_by":{"type":"x_content_parse_exception","reason":"[1:2] [script] unknown field [description]"}},"status":500}
2021-09-06T05:33:00.539+0400 INFO cfgfile/reload.go:224 Loading of config files completed.
2021-09-06T05:33:00.539+0400 INFO [input.harvester] log/harvester.go:309 Harvester started for file. {"input_id": "3af869ef-e361-4497-8141-581e50000bc9", "source": "/var/log/nginx/error.log", "state_id": "native::2098665-64772", "finished": false, "os_id": "2098665-64772", "old_source": "/var/log/nginx/error.log", "old_finished": true, "old_os_id": "2098665-64772", "harvester_id": "09bcaedc-744d-49d7-b884-d5ae8c7ce487"}
2021-09-06T05:33:00.539+0400 INFO [input.harvester] log/harvester.go:309 Harvester started for file. {"input_id": "3f533b20-703e-4018-80c9-81c214c0cac5", "source": "/var/log/nginx/access.log", "state_id": "native::2098662-64772", "finished": false, "os_id": "2098662-64772", "old_source": "/var/log/nginx/access.log", "old_finished": true, "old_os_id": "2098662-64772", "harvester_id": "fc624524-2349-4327-94be-1b6971a41338"}
2021-09-06T05:33:03.490+0400 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:101 add_cloud_metadata: hosting provider type not detected.

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 7, 2021, 3:02am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/2 "2021-09-07T03:02:30Z")

</div>

Yes the matrix says that and that is the intent but it looks like this slipped through. It appears that the description config doesn't exist for the script processor in ES 7.8. You'll either have to upgrade or remove the description fields. I'd open a GitHub issue to report it so it can be fixed.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [September 7, 2021, 3:14am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/3 "2021-09-07T03:14:07Z")

</div>

Hi @Mary_Cane_Bandohan Welcome the community

Actually if you look closely.. the comparability reads.

> ^ Elasticsearch output compatibility - Beats, Logstash, and Elastic Agent indexing data to Elasticsearch

Meaning 7.14 beats are compatible as an output destination (i.e filebeat can write to Elasticsearch), it does not say that all functionality is compatible... in this This case as @legoguy1000 points out the ngnix module has evolved and is using new fields / capabilities that did not exist in 7.8.1.

You should perhaps consider upgrading elasticsearch.

Also in general I would consider it best practices to keep elasticsearch "ahead" of beats as recommended in the upgrade guide [here](https://www.elastic.co/guide/en/elastic-stack/current/upgrading-elastic-stack.html#upgrade-order-elastic-stack)

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 7, 2021, 12:09pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/4 "2021-09-07T12:09:41Z")

</div>

@andrewkroh is this something that you guys want to fix? It'd be an easy add to the compatibility function.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [September 7, 2021, 12:35pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/5 "2021-09-07T12:35:39Z")

</div>

We have an issue open to clarify what the support matrix should say. [https://github.com/elastic/beats/issues/26629](https://github.com/elastic/beats/issues/26629)

I wasn't aware of the `description` field being new so yes, @legoguy1000, I think we should add a rewrite function to handle this case too. This will enable filebeat to automatically rewrite the pipeline without the `description` field so that it should work on 7.8 (assuming there are no other incompatibilities). The oldest we test with is ES 7.11.

@Mary_Cane_Bandohan, as an immediate workaround you could try removing the `description: ` lines from the pipelines then try again. Those are in `/usr/share/filebeat/module/*/*/ingest/*.yml` You only need to fix the modules you are using like `/usr/share/filebeat/module/nginx/access/ingest/pipeline.yml`.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 7, 2021, 1:37pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/6 "2021-09-07T13:37:41Z")

</div>

@andrewkroh looks like the description field was added in 7.12 for all processors. Should be easy enough to update the function.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [September 7, 2021, 1:58pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/7 "2021-09-07T13:58:57Z")

</div>

I think it was first added in v7.9.0 (docs might have been updated later). Otherwise the [7.11 filebeat module tests](https://github.com/elastic/beats/blob/8afb80e6fa06fc9fce5fff1378b4756de90e647a/filebeat/Jenkinsfile.yml#L47-L52) should have caught it.

> <https://github.com/elastic/elasticsearch/pull/57906>
>
> This commit adds an optional field, \`description\`, to all ingest processors
> so …that users can explain the purpose of the specific processor instance.
> 
> Closes #56000.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 7, 2021, 2:12pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/8 "2021-09-07T14:12:38Z")

</div>

Yup, looking at the elasticsearch github Java code, that appears to be the case. PR submitted [https://github.com/elastic/beats/pull/27774](https://github.com/elastic/beats/pull/27774).

---

<div class="post-metadata">

**Author:** ![Mary\_Cane\_Bandohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mary_cane_bandohan/32/81420_2.png) [@Mary\_Cane\_Bandohan](https://discuss.elastic.co/u/Mary_Cane_Bandohan)\
**Post date:** [September 10, 2021, 11:57pm UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/10 "2021-09-10T23:57:01Z")

</div>

Hi Team, just an update.  
We have removed the `description:` lines in `pipeline.yml` as suggested and it fixed the problem.  
Thank you all for your inputs.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 11, 2021, 12:11am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/11 "2021-09-11T00:11:58Z")

</div>

The fix will be in 7.16 to remove it automatically

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2021, 2:12am UTC](https://discuss.elastic.co/t/filebeat-v7-14-0-error-in-loading-pipeline/283407/12 "2021-10-09T02:12:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
