Updating index.max_shingle_diff

When I try to create an index I get this message:

ServerError: Type: illegal_argument_exception Reason: "In Shingle TokenFilter the difference between max_shingle_size and min_shingle_size (and +1 if outputting unigrams) must be less than or equal to: [3] but was [4]. This limit can be set by changing the [index.max_shingle_diff] index level setting."

How can I update index.max_shingle_diff ?

I tried
http:///_all/_settings

{
    "index" : {
        "max_shingle_diff" : 10
    }
}

But gives:

{
  "error": {
    "root_cause": [
      {
        "type": "index_not_found_exception",
        "reason": "no such index [null] and no indices exist",
        "resource.type": "index_expression",
        "resource.id": "_all"
      }
    ],
    "type": "index_not_found_exception",
    "reason": "no such index [null] and no indices exist",
    "resource.type": "index_expression",
    "resource.id": "_all"
  },
  "status": 404
}

Hi @Benjamin1

Try this:

PUT INDEX_NAME/_settings
{
    "index" : {
        "max_shingle_diff" : 10
    }
}

Hi @RabBit_BR
What should I put for INDEX_NAME ? I have no index created. I get the error message when I try to create new index.

ServerError: Type: illegal_argument_exception Reason: "In Shingle TokenFilter the difference between max_shingle_size and min_shingle_size (and +1 if outputting unigrams) must be less than or equal to: [3] but was [4]. This limit can be set by changing the [index.max_shingle_diff] index level setting."

Sorry, I thought the index already existed.

Well I believe you will need to change the max_shingle_diff in phases.

First I created the index in the valid range (ex: min=2 and max=3), then you will close the index and run the command I mentioned above, change the index_name to the name of the index you created.
Now you will run the command to update the _setting with the shingle interval you want.
Then you will open the index.

Create Index
PUT /my-index-000001
{
  "settings": {
    "analysis": {
      "analyzer": {
        "en": {
          "tokenizer": "standard",
          "filter": [ "my_shingle_filter" ]
        }
      },
      "filter": {
        "my_shingle_filter": {
          "type": "shingle",
          "min_shingle_size": 2,
          "max_shingle_size": 3,
          "output_unigrams": false
        }
      }
    }
  }
}


Close Index
POST my-index-000001/_close


Change Max Shingle Diff
PUT my-index-000001/_settings
{
    "index" : {
        "max_shingle_diff" : 8
    }
}

Update Shingle Max Min Size
PUT my-index-000001/_settings
{
  "analysis": {
    "analyzer": {
      "en": {
        "tokenizer": "standard",
        "filter": [
          "my_shingle_filter"
        ]
      }
    },
    "filter": {
      "my_shingle_filter": {
        "type": "shingle",
        "min_shingle_size": 2,
        "max_shingle_size": 6,
        "output_unigrams": false
      }
    }
  }
}

Open Index
POST my-index-000001/_open

Hi @RabBit_BR
Thanks.

I found easier way.

During index creation:

PUT /test
{
  "settings": {
    "number_of_shards": 1,
     "max_shingle_diff" : 10
  },
  "mappings": {
    "properties": {
      "field1": { "type": "text" }
    }
  }
}
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.