# Bug in bulk requests with mix of requests with and without pipelines

**URL:** <https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973>\
**Category:** Elasticsearch\
**Created:** [July 28, 2020, 8:34pm UTC](https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973 "2020-07-28T20:34:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sihtam23](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sihtam23/32/127353_2.png) [@sihtam23](https://discuss.elastic.co/u/sihtam23)\
**Post date:** [July 28, 2020, 8:34pm UTC](https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973/1 "2020-07-28T20:34:38Z")

</div>

I'm new here but not new to using the Elastic stack; I apologise if I get reporting this bug wrong.

This bug exists in Elasticsearch 7.8. I know it doesn't exist in 6.8 but I haven't looked at all the in-between versions.

If I put together a bulk request with a mix of indexing requests with some that call an ingest pipeline and some that don't then there exists a problem if either the pipeline fails or the pipeline contains a drop processor. The handling of the request plays around with the wrong "slot" number in the bulk request / response as it counts only requests with pipelines to figure out that "slot" for the callback in the "executeBulkRequest" method of "IngestService". This results in the wrong items being marked as failed or dropped. I haven't played with failures so much so that needs verifying but here are some steps to reproduce for the drop processor.

```auto
 curl -s -XPUT -H Content-type:application/json localhost:9200/_ingest/pipeline/mypipeline -d '{ "processors": [{ "drop": { } }] }'

```

My bulk request...

```auto
{"index":{"_index":"myindex","_type":"_doc","_id":"1"}}
{"test":"1"}
{"index":{"_index":"myindex","_type":"_doc","_id":"2","pipeline":"mypipeline"}}
{"test":"2"}

```

```auto
curl -s -XPOST -H Content-type:application/json localhost:9200/_bulk --data-binary @bulk_request

```

The response...

```auto
{
  "took": 242,
  "ingest_took": 11,
  "errors": false,
  "items": [
    {
      "index": {
        "_index": "myindex",
        "_type": "_doc",
        "_id": "1",
        "_version": -3,
        "result": "noop",
        "_shards": {
          "total": 0,
          "successful": 0,
          "failed": 0
        },
        "status": 200
      }
    },
    {
      "index": {
        "_index": "myindex",
        "_type": "_doc",
        "_id": "2",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}

```

And if you search the index it contains the wrong document...

```auto
{
  "took": 55,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "myindex",
        "_type": "_doc",
        "_id": "2",
        "_score": 1,
        "_source": {
          "test": "2"
        }
      }
    ]
  }
}

```

I believe the fix should be relatively simple in IngestService but maybe I'm missing something.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 29, 2020, 11:05pm UTC](https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973/2 "2020-07-29T23:05:09Z")

</div>

Welcome to our community! 😃

If you think this is a bug, please create a new issue in [https://github.com/elastic/elasticsearch/issues/](https://github.com/elastic/elasticsearch/issues/).

---

<div class="post-metadata">

**Author:** ![sihtam23](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sihtam23/32/127353_2.png) [@sihtam23](https://discuss.elastic.co/u/sihtam23)\
**Post date:** [July 30, 2020, 5:53am UTC](https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973/3 "2020-07-30T05:53:11Z")

</div>

Thanks Mark.

I've opened a bug at [https://github.com/elastic/elasticsearch/issues/60437](https://github.com/elastic/elasticsearch/issues/60437).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 27, 2020, 5:53am UTC](https://discuss.elastic.co/t/bug-in-bulk-requests-with-mix-of-requests-with-and-without-pipelines/242973/4 "2020-08-27T05:53:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.