Failure: [lang_ident_neural_network] model could not find non-null numerical array named [embedding_vector]

[ccat (Charlotte)]

Hi,
I am running an inference processor for language identification on a text field (model ID: lang_ident_model_1).

• The processor works when I test it with sample data using the _ingest REST endpoint with _simulate.

• The processor also works on a short text field (an email subject) in my actual data, which I tested by reindexing the documents.

• The processor fails on a longer text field (an email body) in my actual data. It has exactly the same text analyzers configured as the subject field and I can view it in Kibana and perform text search successfully. I captured the failure message in the pipeline and it reads:

[lang_ident_neural_network] model could not find non-null numerical array named [embedding_vector]

I am running Elasticsearch 7.12.0 on a two node cluster. The cluster looks healthy and I see no errors in the logs.
What could cause such a failure?

[ccat (Charlotte)]

I just realised that the problem is probably with the way I am testing my pipeline.
I am reindexing from an existing index and the inference is failing on a text field that was excluded from _source in the original mapping.
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#include-exclude