Array definition in XContentBuilder Causes Exception

Elastic Stack Elasticsearch
1

All: First, I plead guilty for being naive about setting up a mapping for an ES Index.

Ultimately I would like to set up an aggregation on an array that I have defined in my mapping. See "pathWithType" here:

                XContentBuilder builder = jsonBuilder()
                    .startObject()
                        .startObject(XXXXXX)
                            .startObject("properties")
                                .startObject("latlon").field("type", "geo_point").endObject()
                                .startObject("path").field("type", "text").field("analyzer", "path-analyzer")
                                                    .field("fielddata", true).endObject()

                                .startArray("pathWithType")
                                    .startObject("path").field("type", "text").field("analyzer", "path-analyzer")
                                                        .field("fielddata", true).endObject()
                                    .startObject("fullPath").field("type", "keyword").field("index", "true").endObject()
                                    .startObject("pathType").field("type", "text").field("index", "true").endObject()
                                .endArray()

                                .startObject("emailString").field("type", "text").field("analyzer", "url-email-
                                     analyzer").endObject()
                                .startObject("fullPath").field("type", "keyword").field("index", "true").endObject()
                                .startObject("statesKW").field("type", "text")
                                                        .field("analyzer", "keyword").field("fielddata", true).endObject()
                                .startObject("audiences").field("type", "text")
                                                            .field("analyzer", "keyword").field("fielddata", true).endObject()
                                .startObject("orgTypes").field("type", "text").field("analyzer", "keyword")
                                                        .field("fielddata", true).endObject()
                                .startObject("suggest").field("type", "completion").endObject()
                                .startObject("linkUrl").field("type", "text").field("analyzer", 
                                         "domain_name_analyzer").endObject()
                                .startObject("linkText").field("type", "text").field("analyzer", 
                                   "domain_name_analyzer").endObject()
                                .startObject("details").field("type", "text").field("analyzer", 
                                      "domain_name_analyzer").endObject()
                                .startObject("legalName").field("type", "text").endObject()
                                .startObject("pathType").field("type", "text").endObject()
                            .endObject()
                        .endObject()
                    .endObject();

With this mapping I get a successful build of the index using the BulkProcessor.

However, with the "pathWithType" array defined in this mapping, when my application runs a search against the index ES returns:

Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]

Obviously the source of this issue must have to do with the array definition in that XContentBuilder mapper. Can somebody please enlighten to what is probably a simple error I have in my mapping?

And also, would I be able to create an aggregation against the "pathWithType" array, like this:
AggregationBuilder subjectAggWithType = AggregationBuilders.terms("subjectWithType").field("pathWithType"))

... or some such syntax?

As always, thank you for your time and assisance.
Gary

2

Please make the effort to format your post to be as readable as possible (in particular, wrap code in preformatted blocks) - there's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.

3

Can you please post the actual mapping (using Get Mapping) rather than the XContent code.

If the problem really is the mapping then it will be easier to spot problems from the JSON rather than the Java that built it.

What sort of search? Does the problem occurr if you simply run GET /index/_search or does it depend on the query?

What do the ES logs show?

4

Tim:

I'm sorry about the sloppiness. Let me try again.

We have over 26,500 documents in our index. A request has come along to separate those documents into two types for a bit of a more specific display to our customer.

To do this we opened up a new field in the index that will allows us to separate the two different types of documents. For simplicity sake lets just say we now have type "1" and type "2".

So now I am trying to retrieve from the index a block of type "1" documents and then a block of type "2" documents. I am not sure how to accomplish this. I was thinking of using aggregations with a subaggregation.

First here is the mapping using the Get Mapping (Because it is quite long I have deleted irrelevant definitions):

{
"XXX": {
"mappings": {
  "resource": {
    "properties": {
		...
		More definitions removed
		...
      "id": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
		...
		More definitions removed
		...

      "path": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },

      "pwtPath": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "pwtPathType": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
		...
		More definitions follow
		...
}

}
}

When the index builder program runs (a java class) This mapping throws this exception:

com.fasterxml.jackson.core.JsonGenerationException: Current context not Object but root

.. It is after I added the "pathWithType..." definition to the XContentBuilder code in the index builder program this exception started to be thrown. In case it helps, this is the XContentBuilder code for the "pathWithType" definition:

.startObject("pathWithType").field("type", "nested")
    .startObject("properties")
       .startObject("pwtPathType").field("type", "text")
                                  .field("index", "true").endObject()
       .startObject("pwtPath").field("type", "text")
                                  .field("analyzer", "path-analyzer")
                                  .field("fielddata", true).endObject().endObject()
    .endObject()
.endObject()

..however the index does get built using the BulkProcessor.builder which is implemented in the index builder program. But a bigger problem is that now when the java code runs a search using this index (the java code builds a large BoolQuery. Its very large. If you want to see it please let me know) I get this exception thrown (contains a long reason):

Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]
ElasticsearchStatusException[Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]];
nested: ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception,
reason=Fielddata is disabled on text fields by default. Set fielddata=true on [path] in order to load fielddata
in memory by uninverting the inverted index.

interestingly enough I do get the same error when I run the identical query in Kibana. That same message about "Fielddata is disabled on text fields by default. Set fielddata=true on [path] in order to load fielddata..." This does not make sense because in the XContentBuilder "fielddata" for "path" is certainly set to true:

.startObject("path").field("type", "text").field("analyzer", "path-analyzer")
                                         .field("fielddata", true).endObject()

So that is one problem that I can't figure out.

Now each document contain a "pwtPathType" and a "pwtPath" data elements needed to hopefully run some kind of a aggregation.

See here:

     {
    "_index": "XXX",
    "_type": "resource",
    "_id": "14485405",
    "_score": 10.4921665,
    "_source": {
      "id": "14485405",
		....
      "pwtPathType": "1",
		...
		...
		...
      "pwtPath": [
        [
          "Health/Medical, Psychological & Behavioral Health Conditions/Traumatic Brain Injury (TBI)"
        ]
      ],

OK.. I tried to keep it clean and short but I wanted to provide all relevant information. And maybe there are a couple of issues here. Anyway any assistance you can provide as to how to get documents that are aggregated by "pwtPathType like:

         documents with pwtPathType = 1:
              pwtPath values for all documents whos pwtPathType = 1
        documents with pwtPathType = 2:
              pwtPath values for all documents whos pwtPathType = 2

Thanks again. Gary

5

Upon further wrestling with this I managed to get around the "com.fasterxml.jackson.core.JsonGenerationException: Current context not Object but root" exception by changing up the definition of my nested data in the XContentBuilder to this:

	.startObject("pathWithType")
	.startObject("properties")
		.startObject("pwt").field("type", "nested")
			.startObject("properties")
				.startObject("pwtPathType").field("type", "text").field("index", "true")
							.field("fielddata", true)
				.endObject()
				.startObject("pwtPath").field("type", "text").field("analyzer", "path-analyzer")
							.field("fielddata", true)
				.endObject()
			.endObject()
		.endObject()
	.endObject()
.endObject()

That definition resulted in this in the mappings:

	"pathWithType": {
	"type":"nested",
	"properties": {
		"pwtPathType": {
			"type": "text", "index": "true"
		}
		,
		"pwtPath": {
			"type": "text", "analyzer": "path-analyzer"
		}
	}
}

I noticed that while I defined "fieldadata" in both the "pwtPathType" and "pwtPath" objects, the "fielddata" attribute does not appear in the "pathWithType" Get Mappings results.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.