I am running an inference processor for language identification on a text field (model ID: lang_ident_model_1).
The processor works when I test it with sample data using the _ingest REST endpoint with _simulate.
The processor also works on a short text field (an email subject) in my actual data, which I tested by reindexing the documents.
The processor fails on a longer text field (an email body) in my actual data. It has exactly the same text analyzers configured as the subject field and I can view it in Kibana and perform text search successfully. I captured the failure message in the pipeline and it reads:
[lang_ident_neural_network] model could not find non-null numerical array named [embedding_vector]
I am running Elasticsearch 7.12.0 on a two node cluster. The cluster looks healthy and I see no errors in the logs.
What could cause such a failure?