InferenceConfig doesn't support text_expansion value when creating a pipeline from java

Elastic Stack Elasticsearch
1

When creating a PutPipelineRequest with the Java client version 8.9.1, InferenceConfig doesn’t support “text_expansion” value, an error is thrown co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.ingest.InferenceConfig: Unknown field 'text_expansion' (JSON path: processors[0].inference.inference_config.text_expansion)
If the pipeline is created with Elastic Api following the steps from the documentation there is no error :slightly_smiling_face:. Maybe this is a bug in the java client, and it hasn't been updated yet.
Here is the code I used to create the pipeline programmatically in java:

InputStream input = new ClassPathResource("elser_pipeline_config.json").getInputStream();

PutPipelineRequest request = PutPipelineRequest.of(pl->pl
                    .id("my_pipeline_test")
                    .withJson(input)
            );
elasticsearchClient.ingest().putPipeline(request);

// elser_pipeline_config.json :
{
  "description":"My test elser pipeline",
  "processors": [
    {
      "inference": {
        "model_id": ".elser_model_1",
        "target_field": "ml",
        "field_map": { 
          "text": "text_field"
        },
        "inference_config": {
          "text_expansion": { 
            "results_field": "tokens"
          }
        }
      }
    }
  ]
}

What can I do to create the pipeline from Java?
If I create it with the Low Level Client I'll have to have two active clients (Low and High level clients), that would not be optimal.
Thank you.

2

I'm not answering on the main question but I guess @swallez will do. :blush:

To create the "rich" Java Client, you need to create a Low Level one. So there's no overhead by calling the low level instance instead.

3

I hope @swallez have a way to address the error. It's better to make it work with the high level java client to avoid deserialization work and object creation.
InferenceConfig currently only supports classification and regression types.

4

Just to make sure. We don't say high level java client for the new Java API Client :wink:

May be open an issue at GitHub - elastic/elasticsearch-java: Official Elasticsearch Java Client?

5

I'm posting my temporal solution as this can take a while to get fixed.
Taking the suggestion that @dadoonet gave me, I'm using the same RestClient instance needed to create the ElasticsearchClient to perform the Rest request.
Step1: Create the RestClient as specified in the documentation.
Step2: Create the Request and set in its body the pipeline settings.

public boolean createUpdatePipeline(){
Request request = new Request("PUT", "/_ingest/pipeline/my_test_pipeline");
String elserPipeline = 
"{\n" +
            "  \"processors\": [\n" +
            "    {\n" +
            "      \"inference\": {\n" +
            "        \"model_id\": \".elser_model_1\",\n" +
            "        \"target_field\": \"ml\",\n" +
            "        \"field_map\": { \n" +
            "          \"text\": \"text_field\"\n" +
            "        },\n" +
            "        \"inference_config\": {\n" +
            "          \"text_expansion\": { \n" +
            "            \"results_field\": \"tokens\"\n" +
            "          }\n" +
            "        }\n" +
            "      }\n" +
            "    }\n" +
            "  ]\n" +
            "}";
request.setJsonEntity(elserPipeline);
Response response = elasticRestClient.performRequest(request); // elasticRestClient is from step1.
if(response.getStatusLine().getStatusCode() == 200){
                ObjectMapper objectMapper = new ObjectMapper();
                String acknowledged = EntityUtils.toString(response.getEntity());
                AcknowledgedResponse ak_response = objectMapper.readValue(acknowledged, CreatePipelineResponse.class);
                return ak_response.acknowledged(); // create the CreatePipelineResponse class implementing the AcknowledgedResponse interface from Elastic, that way when the error is fixed it won't require many code changes.
            } else return false;
}

I posted this solution in case someone else runs into the same problem as I did.
Do I need to create a new bug so this error can be seen by the Elasticsearch/java team?

6

Thanks for sharing your solution.

Indeed, it'd be great to write all that in a new issue with the workaround. Thanks!

7

I created the issue with the workaround solution. It should be noted that making a text_expansion query is not supported by the Java API Client yet, version 8.10.1.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.