Search template creates unnecessary empty strings in arrays

Elastic Stack Elasticsearch
1

Hi,

why does elastic (mustache) create a empty strings in search templates?

I've got sample data as follows:

Mappings:

PUT /movie_db
{"mappings": {"movie": {"properties": {"title": {"type": "string", "analyzer": "snowball"}, "actors": {"type": "string", "position_offset_gap" : 100, "analyzer": "standard"}, "genre": {"type": "string", "index": "not_analyzed"}, "release_year": {"type": "integer", "index": "not_analyzed"}, "description": {"type": "string", "analyzer": "snowball"}}}}}

Sample data:

POST movie_db/movie
{"title":"Hackers","release_year":1995,"genre":["Action","Crime","Drama"],"actors":["Johnny Lee Miller","Angelina Jolie"],"description":"High-school age computer expert Zero Cool and his hacker friends take on an evil corporation's computer virus with their hacking skills."}

I've added couple of duplicate records, just with different release_year values.

Now when I create search template as follows:

POST /_search/template/testTemplate
{
  "template": {
    "query": {
      "bool": {
        "must": [
          {
            "terms": {
              "release_year": ["{{#release_year}}","{{.}}","{{/release_year}}"]
            }
          }
        ]
      }
    }
  }
}

And query it:

GET /movie_db/_search/template
{
  "template": {
    "id": "testTemplate"
  },
  "params": {
    "release_year": ["1995","1998"]
  }
}

I'm getting an error NumberFormatException. Mustache generates empty strings, which cause it.

{
   "error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[BiAcX5l7SfmwqBthXJ6S5g][movie_db][0]: SearchParseException[[movie_db][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"release_year\":[\"\",\"1\",\"\",\"2\",\"\"]}}]}}}]]]; nested: NumberFormatException[For input string: \"\"]; }{[BiAcX5l7SfmwqBthXJ6S5g][movie_db][1]: SearchParseException[[movie_db][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"release_year\":[\"\",\"1\",\"\",\"2\",\"\"]}}]}}}]]]; nested: NumberFormatException[For input string: \"\"]; }{[BiAcX5l7SfmwqBthXJ6S5g][movie_db][2]: SearchParseException[[movie_db][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"release_year\":[\"\",\"1\",\"\",\"2\",\"\"]}}]}}}]]]; nested: NumberFormatException[For input string: \"\"]; }{[BiAcX5l7SfmwqBthXJ6S5g][movie_db][3]: SearchParseException[[movie_db][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"release_year\":[\"\",\"1\",\"\",\"2\",\"\"]}}]}}}]]]; nested: NumberFormatException[For input string: \"\"]; }{[BiAcX5l7SfmwqBthXJ6S5g][movie_db][4]: SearchParseException[[movie_db][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"release_year\":[\"\",\"1\",\"\",\"2\",\"\"]}}]}}}]]]; nested: NumberFormatException[For input string: \"\"]; }]",
   "status": 400
}

NumberFormatException[For input string: ""]
{"release_year":["","1","","2",""]}

It looks like this query is rendered and empty string cannot be converted into number:

{
  "query": {
    "bool": {
      "must": [
        {
          "terms": {
            "release_year": ["","1995","1998",""]
          }
        }
      ]
    }
  }
}

How to avoid/fix that? I was following documentation. I've asked this question on SO, but solution is quite tricky/hacky

Please advice.

2

The reply from stack overflow is correct. You should follow the advice from there and set your template and an escaped JSON string.

Also, once you template is an escaped JSON string you should be able to set release_year by just passing in the array directly. something like this:

"release_year": {{release_year}}

But the template will definitely need to be an escaped JSON string otherwise the request will not the valid JSON and you will get parse errors.

3

I could've written that before. Here it is:

https://www.elastic.co/guide/en/elasticsearch/reference/1.6/search-template.html#_passing_an_array_of_strings

4

This code that you have is saying 'for each element in the array, prepend "," and append a ","' this means that after processing the first array entry, the value will be ",1995," then after the second entry it will be ",1995,,1998,". The mustache script is doing exactly what you are asking it to do. Where in the documentation are you seeing this?