# Ingest Pipeline Failure Processor Shard Failures

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541>\
**Category:** Kibana\
**Created:** [April 21, 2023, 7:12pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541 "2023-04-21T19:12:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [April 21, 2023, 7:12pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/1 "2023-04-21T19:12:39Z")

</div>

I set up an ingest pipeline in Dev Tools console with this command

```auto
PUT _ingest/pipeline/ams-log-pipeline
{
  "processors": [
    {
      "dissect": {
        "field": "message",
        "pattern": "%{@timestamp} %{logLevel} %{className} %{messageContent} %{}",
        "if": "ctx?.fields?.app_id != null && ctx.fields.app_id != 'ams-kafka-consumer'",
        "ignore_failure": true
      }
    },
    {
      "dissect": {
        "field": "message",
        "pattern": "%{logDate} %{logTime} %{logLevel} %{className} %{task} CACHE_MANAGER_RESPONSE %{kafkaTopic} %{statusCode} %{statusString}",
        "if": "ctx?.fields?.app_id != null && ctx.fields.app_id == 'ams-kafka-consumer'",
        "ignore_failure": true
      }
    }
  ],
  "on_failure": [
    {
      "set": {
        "description": "Record error information",
        "field": "error_information",
        "value": "Processor {{ _ingest.on_failure_processor_type }} with tag {{ _ingest.on_failure_processor_tag }} in pipeline {{ _ingest.on_failure_pipeline }} failed with message {{ _ingest.on_failure_message }}"
      }
    }
  ]
}

```

The ingest pipeline appears to let through documents that its 2 processors cannot deal with instead of just failing and letting nothing through - this is good for us.

However, I see shard failures when I run queries in Kibana on documents that were ingested through this pipeline. The failure processor is not populating the field `error_information`, which I understand should have details on what went wrong with the 2 Dissect processors.

I know what an illegal argument exception is but in this case I just can't figure out what is throwing it

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e4f66ff15382ca9dc037529440cf8c279a514ee.png)

What do I need to do to fix this and eliminate the shard failures?

---

<div class="post-metadata">

**Author:** ![jsanz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsanz/32/53734_2.png) [@jsanz](https://discuss.elastic.co/u/jsanz)\
**Post date:** [May 4, 2023, 1:57pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/2 "2023-05-04T13:57:14Z")

</div>

Have you tried the Simulate API with a failing sample document to check the appearance of the resulting document?

> **[Simulate pipeline API | Elasticsearch Guide \[8.7\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/simulate-pipeline-api.html)**

Where are those shards failures happening? Is it on Discover or a Visualization?

---

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [May 4, 2023, 2:21pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/3 "2023-05-04T14:21:01Z")

</div>

Thanks Jorge!

The failures were on a dashboard that included visualizations.

I clicked the Request and Response tabs and found the illegal\_argument\_exception was caused by one of the text fields in the document.

The document had been transformed by our ingest pipeline. The text field was one of the fields generated by a Dissect processor in the pipeline. The same field was being used by a Top Values function in one of the visualizations The error message recommended replacing  
"fieldname" with "fieldname.keyword" as one of the fixes. I did that and the shard failures disappeared.

---

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [May 4, 2023, 2:53pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/4 "2023-05-04T14:53:33Z")

</div>

One last question (I hope) on this thread.

We are using one Filebeat instance to ship logs from six different log files.

This instance is shipping the logs to one ingest pipeline, with 4 Dissect processors to extract fields and add them to applicable documents.

From what I can tell, only one pipeline at a time can be configured in the `output.elasticsearch` section of filebeat.yml. Is this correct?

---

<div class="post-metadata">

**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\
**Post date:** [May 31, 2023, 9:25pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/5 "2023-05-31T21:25:18Z")

</div>

Hi @paolovalladolid

If you use the `pipelines` option in the `elasticsearch.output` section, you can specify an array of piepline selectors: [Configure the Elasticsearch output | Filebeat Reference [8.8] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html#pipeline-option-es)

I hope that helps!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2023, 9:25pm UTC](https://discuss.elastic.co/t/ingest-pipeline-failure-processor-shard-failures/330541/6 "2023-06-28T21:25:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.