# Percolator throws INTERNAL\_SERVER\_ERROR on irrelevant field

**URL:** <https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311>\
**Category:** Elasticsearch\
**Created:** [July 12, 2016, 2:51pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311 "2016-07-12T14:51:08Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 12, 2016, 2:51pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/1 "2016-07-12T14:51:08Z")

</div>

Hello, I just upgraded from version 1.4.4 to 2.3.3 and I'm having an issue with the percolator:

Before, we used to send the percolator requests using the \_mpercolate endpoint for an existing document as follows:

```
curl -XGET 'localhost:9200/_mpercolate' -d '
{"percolate":{"id":"myid_123456","index":"week_29","percolate_index":"my_percolator_index","type":"article"}}
{}
'

```

Where the article with ID "myid\_123456" was indexed on index "week\_29" and has many attributes as part of the document. The percolator index, "my\_percolator\_index" only has a query to match two of the fields in the article document:

```
{
  "_index": "my_percolator_index",
  "_type": ".percolator",
  "_id": "897987987",
  "_version": 1,
  "_score": 1,
  "_source": {
    "query": {
      "bool": {
        "must": {
          "query_string": {
            "query": "+(title:Obama body:Obama) +(title:Clinton body:Clinton)"
          }
        }
      }
    },
    "unique_query_name": "candidates",
    "unique_query_id": 512508281
  }
}

```

The response in 1.4.4 was successful, returning the ID of the percolator query that match the document if title or body has a match. However, in 2.3.3 the response throws an error for any other field in the document that does not have a analyzer in the mapping, which for us is not needed:

```
{
  "responses": [
    {
      "took": 9,
      "_shards": {
        "total": 2,
        "successful": 0,
        "failed": 2,
        "failures": [
          {
            "shard": 0,
            "index": "my_percolator_index",
            "status": "INTERNAL_SERVER_ERROR",
            "reason": {
              "type": "exception",
              "reason": "Failed to create token stream for [author]",
              "caused_by": {
                "type": "illegal_argument_exception",
                "reason": "Field [author] has no associated analyzer"
              }
            }
          }
        ]
      },
      "total": 0,
      "matches": [
        
      ]
    }
  ]
}

```

Can I get some help on solving this issue without having to add every field to the mapping template?

Thanks a lot!

Javier C.

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 12, 2016, 5:52pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/2 "2016-07-12T17:52:21Z")

</div>

UPDATE:

I changed the way we index the percolator query to specifically set the fields we want the query to work on:

```
{
  "_source": {
    "query": {
      "bool": {
        "must": {
          "query_string": {
            "query": "Obama Clinton",
            "fields": [
              "title",
              "body"
            ]
          }
        }
      }
    }
  }
}

```

And I still get the same error 😢

Thanks!

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [July 13, 2016, 5:36am UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/3 "2016-07-13T05:36:36Z")

</div>

For some reason the author field doesn't have an analyzer. Can your share the `article` mapping?

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 13, 2016, 12:25pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/4 "2016-07-13T12:25:33Z")

</div>

Hello Martijn, bellow is the information for the template, week\_29 index mapping (where the document is stored), my\_percolator\_index mapping (where the .percolator query is stored) and a sample document:

Custom template:

```
{
  "custom_template": {
    "template": "*",
    "settings": {
      "index": {
        "analysis": {
          "filter": {
            "length_filter": {
              "type": "length",
              "min": "2"
            }
          },
          "analyzer": {
            "custom_analyzer": {
              "type": "custom",
              "filter": [
                "lowercase",
                "stop",
                "length_filter"
              ],
              "tokenizer": "whitespace"
            }
          }
        }
      }
    },
    "mappings": {
      "article": {
        "properties": {
          "body": {
            "analyzer": "custom_analyzer",
            "type": "string"
          },
          "title": {
            "analyzer": "custom_analyzer",
            "type": "string"
          }
        }
      }
    }
  }
}

```

week\_29 index mapping:

```
{
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "author": {
          "type": "string"
        }
      }
    }
  }
}

```

my\_percolator\_index mapping:

```
{
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        }
      }
    },
    ".percolator": {
      "properties": {
        "query": {
          "enabled": false,
          "type": "object"
        },
        "unique_query_name": {
          "type": "string"
        },
        "unique_query_id": {
          "type": "long"
        }
      }
    }
  }
}

```

sample document:

```
{
  "_index": "week_29",
  "_type": "article",
  "_id": "myid_123456",
  "_version": 1,
  "_score": 1,
  "_source": {
    "title": "the title of the article that includes Obama",
    "body": "the body of the article that includes Clinton",
    "author": "Javier"
  }
}

```

**note** : this is similar to what I have in es 1.4.4 version (which works) where the mapping for 'author' is in the index where the document is stored but not in the percolator index.

Let me know if you need any additional information, and thanks a lot for the help!

Javier C.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [July 13, 2016, 12:46pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/5 "2016-07-13T12:46:29Z")

</div>

I think you're getting this error because the `custom_analyzer` analyzer isn't configured in the `my_percolator_index` index. Can you add the analyzer related index settings from index `week29` to `my_percolator_index` index? (you do need to close and open the index for these settings to take in effect)

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 13, 2016, 1:25pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/6 "2016-07-13T13:25:12Z")

</div>

Sorry for the confusion, the my\_percolator\_index index does have the analyzer, I just didn't copy the whole Index Metadata information, but here it is:

```
{
  "state": "open",
  "settings": {
    "index": {
      "creation_date": "1468364676444",
      "uuid": "tFMhPX5XRri-MVc8iEZVeA",
      "analysis": {
        "filter": {
          "length_filter": {
            "type": "length",
            "min": "2"
          }
        },
        "analyzer": {
          "custom_analyzer": {
            "type": "custom",
            "filter": [
              "lowercase",
              "stop",
              "length_filter"
            ],
            "tokenizer": "whitespace"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "2",
      "version": {
        "created": "2030399"
      }
    }
  },
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        }
      }
    },
    ".percolator": {
      "properties": {
        "query": {
          "enabled": false,
          "type": "object"
        },
        "unique_query_name": {
          "type": "string"
        },
        "unique_query_id": {
          "type": "long"
        }
      }
    }
  },
  "aliases": [
    
  ]
}

```

Thanks again!

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 15, 2016, 6:09pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/7 "2016-07-15T18:09:12Z")

</div>

Hi Martijn, do you have any other suggestions?

I still get the error ☹ even when I index a new document to my week\_29 index and it contains a new fields and I try to run the percolator, it fails because of the missing analyzer error.

Thanks for the help!

Javier

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [July 15, 2016, 7:50pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/8 "2016-07-15T19:50:09Z")

</div>

Hi Javier,

I've tried to reproduce your error here with the snippets you provided in this question.  
However Im unable to reproduce it. Can you share a reproduction (minimal steps to reproduce the error)?

This is what I've tried:

```auto
PUT /test
{
  "settings": {
    "index": {
      "creation_date": "1468364676444",
      "uuid": "tFMhPX5XRri-MVc8iEZVeA",
      "analysis": {
        "filter": {
          "length_filter": {
            "type": "length",
            "min": "2"
          }
        },
        "analyzer": {
          "custom_analyzer": {
            "type": "custom",
            "filter": [
              "lowercase",
              "stop",
              "length_filter"
            ],
            "tokenizer": "whitespace"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "2",
      "version": {
        "created": "2030399"
      }
    }
  },
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        }
      }
    },
    ".percolator": {
      "properties": {
        "query": {
          "enabled": false,
          "type": "object"
        },
        "unique_query_name": {
          "type": "string"
        },
        "unique_query_id": {
          "type": "long"
        }
      }
    }
  }
}

PUT /test/.percolator/1
{
  "query": {
    "bool": {
      "must": {
        "query_string": {
          "query": "+(title:Obama body:Obama) +(title:Clinton body:Clinton)"
        }
      }
    }
  }
}

GET test/article/_percolate
{
  "doc": {
    "title": "the title of the article that includes Obama",
    "body": "the body of the article that includes Clinton",
    "author": "Javier"
  }
}

```

But that ran successful here.

Martijn

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 18, 2016, 1:37pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/9 "2016-07-18T13:37:23Z")

</div>

Hi Martijn, thanks for the help.

I noticed that you used the same index (test) for the document and the percolator. However, in my case, I have separate indices. One where I store the document (week\_29), and a separate one where I store the percolator query (my\_percolator\_index). And that's how I get the error. I will post reproducible steps, let me work them out.

Javier.

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 18, 2016, 2:13pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/10 "2016-07-18T14:13:20Z")

</div>

These are the steps to reproduce the problem:

```
curl -XPUT 'http://localhost:9200/week_29/' -d '{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "length_filter": {
            "type": "length",
            "min": "2"
          }
        },
        "analyzer": {
          "custom_analyzer": {
            "type": "custom",
            "filter": [
              "lowercase",
              "stop",
              "length_filter"
            ],
            "tokenizer": "whitespace"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "2",
      "version": {
        "created": "2030399"
      }
    }
  },
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        }
      }
    },
    ".percolator": {
      "properties": {
        "query": {
          "enabled": false,
          "type": "object"
        },
        "unique_query_name": {
          "type": "string"
        },
        "unique_query_id": {
          "type": "long"
        }
      }
    }
  }
}'

curl -XPUT 'http://localhost:9200/my_percolator_index/' -d '{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "length_filter": {
            "type": "length",
            "min": "2"
          }
        },
        "analyzer": {
          "custom_analyzer": {
            "type": "custom",
            "filter": [
              "lowercase",
              "stop",
              "length_filter"
            ],
            "tokenizer": "whitespace"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "2",
      "version": {
        "created": "2030399"
      }
    }
  },
  "mappings": {
    "article": {
      "properties": {
        "body": {
          "analyzer": "custom_analyzer",
          "type": "string"
        },
        "title": {
          "analyzer": "custom_analyzer",
          "type": "string"
        }
      }
    },
    ".percolator": {
      "properties": {
        "query": {
          "enabled": false,
          "type": "object"
        },
        "unique_query_name": {
          "type": "string"
        },
        "unique_query_id": {
          "type": "long"
        }
      }
    }
  }
}'

curl -XPUT 'http://localhost:9200/my_percolator_index/.percolator/1' -d '{
  "query": {
    "bool": {
      "must": {
        "query_string": {
          "query": "+(title:Obama body:Obama) +(title:Clinton body:Clinton)"
        }
      }
    }
  }
}'

curl -XPUT 'http://localhost:9200/week_29/article/99' -d '{
  "title": "the title of the article that includes Obama",
  "body": "the body of the article that includes Clinton",
  "author": "Javier"
}'

curl -XGET 'localhost:9200/_mpercolate' -d '
{"percolate":{"id":"99","index":"week_29","type":"article","percolate_index":"my_percolator_index"}}
{}
'

```

And I get the error:

```
{
  "responses": [
    {
      "took": 14,
      "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 1,
        "failures": [
          {
            "shard": 1,
            "index": "my_percolator_index",
            "status": "INTERNAL_SERVER_ERROR",
            "reason": {
              "type": "exception",
              "reason": "Failed to create token stream for [author]",
              "caused_by": {
                "type": "illegal_argument_exception",
                "reason": "Field [author] has no associated analyzer"
              }
            }
          }
        ]
      },
      "total": 0,
      "matches": [
        
      ]
    }
  ]
}

```

Thanks!

Javier.

---

<div class="post-metadata">

**Author:** ![mvg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mvg/32/98890_2.png) [@mvg](https://discuss.elastic.co/u/mvg)\
**Post date:** [July 20, 2016, 8:14am UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/11 "2016-07-20T08:14:31Z")

</div>

Apologies for the late response.

The reason for this error is that the `author` field is not mapped in the percolator index. Adding the field to the percolate index should fix this.

Unfortunately there is a difference between how percolating an existing document and percolating a provided document handles this situation. When percolating a provided document and field is missing then the percolate api accepts this and even updates the mapping. Whereas when percolating an existing document this is not the case and as you see fails if an unknown field is encountered.

In ES 5.0, the percolator has been rewritten and the differences between two ways of percolating have disappeared and your example (with minor changes to the mapping (making use of the `percolator` field mapping)) works correctly.

---

<div class="post-metadata">

**Author:** ![javierco](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@javierco](https://discuss.elastic.co/u/javierco)\
**Post date:** [July 20, 2016, 12:36pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/12 "2016-07-20T12:36:50Z")

</div>

Thanks for the support Matijn, I understand.

It is unfortunate that this feature that used to work on ES 1.4.4, was removed on ES 2.3.3, to now being put again on ES 5.0.

We just upgraded to v2.3.3 from v.1.4.4 and now have to write every single field mapping into the percolator index manually, even when the document index adds the mapping automatically.

Again, I appreciate your help in finding the issue. Take care!!!

Javier C.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:34pm UTC](https://discuss.elastic.co/t/percolator-throws-internal-server-error-on-irrelevant-field/55311/13 "2017-07-05T22:34:04Z")

</div>