GetTransformResponse unable to parse "source" field

Elastic Stack Elasticsearch
,
1

ELK 7.8.1
Using Springboot 2.2.5
I am able to create a transform using the Kibana Stack Management and using the PUT _transform curl command. However when trying to retrieve the same transform using the High Level REST client, I receive the following error:

Caused by: org.elasticsearch.common.xcontent.XContentParseException: [1:137] [transform_config_source] failed to parse field [query]
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:531) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:541) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:324) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ConstructingObjectParser.parse(ConstructingObjectParser.java:171) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ConstructingObjectParser.apply(ConstructingObjectParser.java:163) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.client.transform.transforms.TransformConfig.lambda$static$1(TransformConfig.java:99) ~[elasticsearch-rest-high-level-client-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(AbstractObjectParser.java:169) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$9(ObjectParser.java:386) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:529) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	... 47 common frames omitted
Caused by: org.elasticsearch.common.xcontent.XContentParseException: unknown named object category [org.elasticsearch.index.query.QueryBuilder]
	at org.elasticsearch.common.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:128) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:385) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder(AbstractQueryBuilder.java:320) ~[elasticsearch-7.8.1.jar:7.8.1]
	at org.elasticsearch.client.transform.transforms.QueryConfig.fromXContent(QueryConfig.java:39) ~[elasticsearch-rest-high-level-client-7.8.1.jar:7.8.1]
	at org.elasticsearch.client.transform.transforms.SourceConfig.lambda$static$1(SourceConfig.java:56) ~[elasticsearch-rest-high-level-client-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(AbstractObjectParser.java:169) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$9(ObjectParser.java:386) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:529) ~[elasticsearch-x-content-7.8.1.jar:7.8.1]
	... 55 common frames omitted
2

Welcome!

What are the dependencies you declared for your project?

Asking because when using springboot with elasticsearch, you need to be explicit with some transitive dependencies as SpringBoot declares a version 6.4...

Basically you can put this in your pom.xml:

<properties>
  <elasticsearch.version>7.9.3<elasticsearch.version>
</properties>

See documentation here: https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-customize-dependency-versions

3

Thanks for the response.

Yes right now have it declared as
<elasticsearch.version>7.8.1</elasticsearch.version>

4

Could you share a bit of your code?

5

Sure. Here is the transform that I created:

{
  "id": "bogus",
  "source": {
    "index": [
      "*bogus*"
    ],
    "query": {
      "bool": {
        "filter": [
          {
            "bool": {
              "should": [
                {
                  "exists": {
                    "field": "items.id"
                  }
                }
              ],
              "minimum_should_match": 1
            }
          },
          {
            "bool": {
              "should": [
                {
                  "exists": {
                    "field": "counter.type"
                  }
                }
              ],
              "minimum_should_match": 1
            }
          }
        ]
      }
    }
  },
  "dest": {
    "index": "bogus"
  },
  "sync": {
    "time": {
      "field": "@timestamp",
      "delay": "5m"
    }
  },
  "pivot": {
    "group_by": {
      "items.id.keyword": {
        "terms": {
          "field": "items.id.keyword"
        }
      },
      "@timestamp": {
        "date_histogram": {
          "field": "@timestamp",
          "calendar_interval": "1d"
        }
      }
    },
    "aggregations": {
      "unique_item_investigations": {
        "filter": {
          "bool": {
            "must": [
              {
                "term": {
                  "counter.type": "test"
                }
              },
              {
                "term": {
                  "event.metadata.double_click": false
                }
              }
            ]
          }
        },
        "aggs": {
          "cardinality_unique_item_investigations": {
            "cardinality": {
              "field": "counter.type.keyword"
            }
          }
        }
      }
    }
  },
  "description": "Bogus",
  "settings": {}
}

Here is the java snippet used to call and retrieve the transform

GetTransformRequest getTransformRequest = new GetTransformRequest(transformName);
        try {
            GetTransformResponse getTransformResponse = client.transform().getTransform(getTransformRequest, RequestOptions.DEFAULT);
            return getTransformResponse.getCount() > 0;
        } catch (ElasticsearchException e) {
            logger.info(String.format("Transform - %s - not found", transformName));
        }
        return false;
1 Like
6

Great thanks. I can reproduce the problem...

I opened

7

Perfect, thank you so much!

8

@Craig_Uss what are you supplying as your NamedXContent serializers in your high level rest client constructor?

The error

unknown named object category [org.elasticsearch.index.query.QueryBuilder]

Indicates that for some reason the appropriate named xcontent serializers were not provided to the RestHighLevelClient object.

Please provide your RestHighLevelClient version information and how you are calling the constructor. That will help narrow this down.

9

Digging around in the code, it seems that

static List<NamedXContentRegistry.Entry> getProvidedNamedXContents() {
        List<NamedXContentRegistry.Entry> entries = new ArrayList<>();
        for (NamedXContentProvider service : ServiceLoader.load(NamedXContentProvider.class)) {
            entries.addAll(service.getNamedXContentParsers());
        }
        return entries;
    }

In a 7.8 high level rest client loads all the xcontent parsers for the included plugins. If the search plugin is not included, it won't be loaded I think.

EDIT:

Here is a way to construct a client that includes the SearchModule xcontent serializers.

class HighLevelClient extends RestHighLevelClient {
       HighLevelClient(RestClient restClient) {
            super(restClient, RestClient::close, new SearchModule(Settings.EMPTY, Collections.emptyList()).getNamedXContents());
        }
    }
10 
class HighLevelClient extends RestHighLevelClient {
       HighLevelClient(RestClient restClient) {
            super(restClient, RestClient::close, new SearchModule(Settings.EMPTY, Collections.emptyList()).getNamedXContents());
        }
    }

This worked out for me! Thank you so much for your help on this!

11

Great. But I do think that end users should not have to care about that.
It should be available OOTB.

12

I agree. What I am having trouble with reconciling is building a Transform. In other API operations, I usually have the option to provide the Json source to build the object. Here I have to create a xContentParser. Something like the below (yet to get it to work, but just to give you an idea)

XContentParser parser = JsonXContent.jsonXContent.createParser(new NamedXContentRegistry((new TransformNamedXContentProvider()).getNamedXContentParsers()), LoggingDeprecationHandler.INSTANCE, transformSource);
        TransformConfig transformConfig = TransformConfig.fromXContent(parser);

Is this the right approach for trying to create a PutTransformRequest? I can open a new thread if needed!

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.