# Circuit\_breaking\_exception when creating an ingest pipeline

**URL:** <https://discuss.elastic.co/t/circuit-breaking-exception-when-creating-an-ingest-pipeline/125630>\
**Category:** Elasticsearch\
**Created:** [March 26, 2018, 3:02pm UTC](https://discuss.elastic.co/t/circuit-breaking-exception-when-creating-an-ingest-pipeline/125630 "2018-03-26T15:02:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [March 26, 2018, 3:02pm UTC](https://discuss.elastic.co/t/circuit-breaking-exception-when-creating-an-ingest-pipeline/125630/1 "2018-03-26T15:02:49Z")

</div>

I'm trying to create an ingest pipeline with a simple `remove` processor that contains a lot of field names to remove from the document (\> 100) prior to indexing. There is no `script` processor in the pipeline and I have left the default `script.max_compilations_rate` setting (i.e. 75/5m)

The pipeline looks like this:

```
PUT _ingest/pipeline/my-pipeline
{
  "description": "Test pipeline",
  "processors": [
    {
      "remove": {
        "field": [
          "field1",
          "field2",
          "field3",
          "field4",
          "field5",
          "field6",
          "field7",
          "field8",
          "field9",
          "field10",
          "field11",
          "field12",
          "field13",
          "field14",
          "field15",
          "field16",
          "field17",
          "field18",
          "field19",
          "field20",
          "field21",
          "field22",
          "field23",
          "field24",
          "field25",
          "field26",
          "field27",
          "field28",
          "field29",
          "field30",
          "field31",
          "field32",
          "field33",
          "field34",
          "field35",
          "field36",
          "field37",
          "field38",
          "field39",
          "field40",
          "field41",
          "field42",
          "field43",
          "field44",
          "field45",
          "field46",
          "field47",
          "field48",
          "field49",
          "field50",
          "field51",
          "field52",
          "field53",
          "field54",
          "field55",
          "field56",
          "field57",
          "field58",
          "field59",
          "field60",
          "field61",
          "field62",
          "field63",
          "field64",
          "field65",
          "field66",
          "field67",
          "field68",
          "field69",
          "field70",
          "field71",
          "field72",
          "field73",
          "field74",
          "field75",
          "field76",
          "field77",
          "field78",
          "field79",
          "field80",
          "field81",
          "field82",
          "field83",
          "field84",
          "field85",
          "field86",
          "field87",
          "field88",
          "field89",
          "field90",
          "field91",
          "field92",
          "field93",
          "field94",
          "field95",
          "field96",
          "field97",
          "field98",
          "field99",
          "field100",
          "field101",
          "field102",
          "field103",
          "field104",
          "field105"
        ],
        "ignore_failure": true
      }
    }
  ],
  "on_failure": [
    {
      "append": {
        "field": "meta.errors",
        "value": "{{ _ingest.on_failure_message }}, {{ _ingest.on_failure_processor_type }}, {{ _ingest.on_failure_processor_tag }}"
      }
    }
  ]
}

```

This command results in the following error:

```
{
  "error": {
    "root_cause": [
      {
        "type": "circuit_breaking_exception",
        "reason": "[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting",
        "bytes_wanted": 0,
        "bytes_limit": 0
      }
    ],
    "type": "general_script_exception",
    "reason": "Failed to compile inline script [field15] using lang [mustache]",
    "caused_by": {
      "type": "circuit_breaking_exception",
      "reason": "[script] Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting",
      "bytes_wanted": 0,
      "bytes_limit": 0
    },
    "header": {
      "processor_type": "remove",
      "property_name": "field"
    }
  },
  "status": 500
}

```

The only reason I think I'm getting a script error here is because the processors are getting translated to painless scripts underneath. Am I correct? The only way I can circumvent this is to increase the `max_compilations_rate` setting.

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [March 27, 2018, 3:51am UTC](https://discuss.elastic.co/t/circuit-breaking-exception-when-creating-an-ingest-pipeline/125630/2 "2018-03-27T03:51:51Z")

</div>

Ok, I've kind of figured it out by looking at the source code for [`RemoveProcessor`](https://github.com/elastic/elasticsearch/blob/master/modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/RemoveProcessor.java) where we can clearly see that scripting is involved when configuring the processor. From there on, the error "makes sense", even though it was not clear at the beginning that ingest processors were implemented using scripting.

Using Mustache scripts for removing fields feels a bit like using a canon to shoot a fly, but there must be some context there, so I'll simply stick with the `max_compilations_rate` setting

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 24, 2018, 3:52am UTC](https://discuss.elastic.co/t/circuit-breaking-exception-when-creating-an-ingest-pipeline/125630/3 "2018-04-24T03:52:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.