# Elasticsearch reindex with ELSER pipeline succeeds but only generates embeddings for fraction of documents - no failures reported

**URL:** <https://discuss.elastic.co/t/elasticsearch-reindex-with-elser-pipeline-succeeds-but-only-generates-embeddings-for-fraction-of-documents-no-failures-reported/386228>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [May 8, 2026, 3:12am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-with-elser-pipeline-succeeds-but-only-generates-embeddings-for-fraction-of-documents-no-failures-reported/386228 "2026-05-08T03:12:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Roland-02](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/roland-02/32/147507_2.png) [@Roland-02](https://discuss.elastic.co/u/Roland-02)\
**Post date:** [May 8, 2026, 3:12am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-with-elser-pipeline-succeeds-but-only-generates-embeddings-for-fraction-of-documents-no-failures-reported/386228/1 "2026-05-08T03:12:28Z")

</div>

I'm reindexing documents with an ELSER inference pipeline to generate embeddings, but only a fraction of documents (usually around half) end up with embeddings despite the reindex completing successfully with no failures.

**Setup:**

- Elasticsearch with ELSER model deployed and running

- Source index: ~1000 documents

- Destination index with `rank_features` mapping for embeddings

- Pipeline: `elser_double_embedding_pipeline`

- 

**Reindex Request:**

```auto
POST /_reindex
{
  "source": {
    "index": "v1214_test_staging_20260506_181845"
  },
  "dest": {
    "index": "v1214_test",
    "pipeline": "elser_double_embedding_pipeline"
  }
}

```

**Task Status (Completed Successfully):**

```auto
{
  "completed": true,
  "task": {
    "description": "reindex from [v1214_test_staging_20260506_181845] to [v1214_test]",
    "status": {
      "total": 977,
      "updated": 977,
      "created": 0,
      "deleted": 0,
      "batches": 2,
      "version_conflicts": 0,
      "noops": 0,
      "retries": {
        "bulk": 2,
        "search": 0
      }
    }
  },
  "response": {
    "total": 977,
    "updated": 977,
    "failures": []
  }
}

```

**Problem:** Despite successful completion, only 450 out of 977 documents have embeddings:

```auto
GET /v1214_test/_count
{
  "query": {
    "bool": {
      "should": [
        {"bool": {"must_not": {"exists": {"field": "name_embedding"}}}},
        {"bool": {"must_not": {"exists": {"field": "content_embedding"}}}}
      ],
      "minimum_should_match": 1
    }
  }
}
// Returns: 527 documents missing embeddings

```

**Pipeline Configuration:**

```auto
GET /_ingest/pipeline/elser_double_embedding_pipeline
{
  "elser_double_embedding_pipeline": {
....
    "processors": [
      {
        "inference": {
          "model_id": ".elser_model_2_linux-x86_64",
          "input_output": [
            {
              "input_field": "content",
              "output_field": "content_embedding"
            },
            {
              "input_field": "name",
              "output_field": "name_embedding"
            }
          ]
        }
      }
    ]
  }
}

```

**What I've Tried:**

1. Running `update_by_query` with pipeline - same result

2. Checking for failures in task response - none reported

3. Verifying source documents have `name` and `content` fields - they do

4. Testing pipeline simulation - works correctly

5. Running reindex multiple times - consistently ~46% success rate

**Questions:**

1. Why would reindex report success but not apply the pipeline to all documents?

2. How can I debug which documents are failing to get embeddings when no failures are reported?

3. Is there a way to force the pipeline to process all documents or identify which ones were skipped?

The task description doesn't show the pipeline name (should show `[elser_double_embedding_pipeline]`), which makes me suspect the pipeline isn't being applied, but the API accepts the parameter without error.

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [May 8, 2026, 1:36pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-with-elser-pipeline-succeeds-but-only-generates-embeddings-for-fraction-of-documents-no-failures-reported/386228/2 "2026-05-08T13:36:21Z")

</div>

Have you checked the elasticsearch logs from all nodes? I'm wondering if something silently failed because the model was not available on some node where it was expected to be (just guessing)? If there is nothing in the logs, it might be worth rerunning [with debug-level logging](https://www.elastic.co/docs/deploy-manage/monitor/logging-configuration/update-elasticsearch-logging-levels) for the `org.elasticsearch.xpack.ml.action` package.

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [May 8, 2026, 2:02pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-with-elser-pipeline-succeeds-but-only-generates-embeddings-for-fraction-of-documents-no-failures-reported/386228/3 "2026-05-08T14:02:16Z")

</div>

Also, could you share your pipeline? Does your inference processor have `ignore_failure` set to true?