# Index 256 out of bounds for length 256

**URL:** https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710
**Category:** Elasticsearch
**Created:** [August 14, 2023, 2:37am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710 "2023-08-14T02:37:44Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![1057888035](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1057888035/32/124586_2.png) [@1057888035](https://discuss.elastic.co/u/1057888035)
#### Post date: [August 14, 2023, 2:37am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/1 "2023-08-14T02:37:44Z")

</div>

Hi Team,  
I am trying to execute api like this in elasticsearch

```auto
GET /_analyze
{
  "tokenizer": "keyword",
  "char_filter": [
    {
      "type": "mapping",
      "mappings": ["is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data is a test data => test"]
    }
  ],
  
  "text": "heart malignant hemangiopericytoma is a test data"
}

```

echo error: "Index 256 out of bounds for length 256",  
This error mabye that the mappings length cannot exceed 256， but my project needs more

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [August 14, 2023, 5:36am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/2 "2023-08-14T05:36:54Z")

</div>

Replacing really long strings is not necessarily what a char filter is designed for. What is the problem you are trying to solve using analysis with this char filter (at a higher level)? Can you please elaborate on the problem and use case and also provide a realistic example of how you want your data analyzed?

---

<div class="post-metadata">

### Author: ![1057888035](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1057888035/32/124586_2.png) [@1057888035](https://discuss.elastic.co/u/1057888035)
#### Post date: [August 14, 2023, 8:21am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/3 "2023-08-14T08:21:24Z")

</div>

I have a project  
" **benzylpenicillin allergy**" is a keyword , this equals " **benzyl penicillin allergy**".

Now, there is a text

```auto
POST /index/_doc
{
    "text": "benzyl penicillin allergy should not be used in tissues with poor blood flow. If allergic symptoms occur (e.g. skin rash, itching, shortness of breath), tell a doctor immediately . Before treatment, a hypersensitivity test should be performed if possible."
}

```

I want this document ,when I search for " **benzylpenicillin allergy**" .

I use standard analyzer, it will be analyze to "benzylpenicillin, allergy" ，  
which is not feasible , be because I still have some document containing "allergy" or "penicillin" .

So, I removed the space between them with the **mappings** , and " **benzylpenicillinallergy**" is equal to " **benzylpenicillinallergy**" use alias filter

Do you have a better way ？ Thanks

---

<div class="post-metadata">

### Author: ![1057888035](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1057888035/32/124586_2.png) [@1057888035](https://discuss.elastic.co/u/1057888035)
#### Post date: [August 14, 2023, 8:38am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/4 "2023-08-14T08:38:28Z")

</div>

more:

" **isoniazide allergy**" is equal to " **inh allergy**" and is equal to " **isonicotinylhydrazide allergy**".

” **superficial mycosis** “ is equal to " **steroid-modified tinea infection**" and is equal to " **piedra**".

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [August 14, 2023, 1:22pm UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/5 "2023-08-14T13:22:55Z")

</div>

What about using synonyms?

> **[Boosting the power of Elasticsearch with synonyms](https://www.elastic.co/blog/boosting-the-power-of-elasticsearch-with-synonyms)**
>
> How to use synonyms and synonym filters in Elasticsearch. Synonyms are a powerful tool for increasing the recall of your search system, but there are many subtleties that are important to know and experiment with, especially in conjunction with...

---

<div class="post-metadata">

### Author: ![1057888035](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1057888035/32/124586_2.png) [@1057888035](https://discuss.elastic.co/u/1057888035)
#### Post date: [August 15, 2023, 3:13am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/6 "2023-08-15T03:13:00Z")

</div>

I tried synonyms but it doesn't solve the problem.

It won't work when my synonym contains spaces.

Like this:  
analysis/ **synonym.txt**

```auto
thiopental allergy,penthiobarbital allergy,pentothiobarbital allergy
d-mannitol allergy,mannitol allergy
cefotaxime allergy
cephalosporin allergy
amodiaquine allergy,camoquin allergy,flavoquine allergy

```

**The result I want is**

```auto
GET test_index/_analyze
{
   "analyzer":"my_analyzer",
   "text": "thiopental allergy is a test"
}

result:

{
  "tokens": [
    {
      "token": "thiopental allergy",
      "start_offset": 0,
      "end_offset": 18,
      "type": "word",
      "position": 0
    },
    {
      "token": "penthiobarbital allergy",
      "start_offset": 0,
      "end_offset": 18,
      "type": "word",
      "position": 0
    },
    {
      "token": "pentothiobarbital allergy",
      "start_offset": 0,
      "end_offset": 18,
      "type": "word",
      "position": 0
    },
    {
      "token": "is",
      "start_offset": 19,
      "end_offset": 21,
      "type": "word",
      "position": 0
    },
    {
      "token": "a",
      "start_offset": 22,
      "end_offset": 23,
      "type": "word",
      "position": 0
    },
    {
      "token": "test",
      "start_offset": 24,
      "end_offset": 28,
      "type": "word",
      "position": 0
    },
  ]
}

```

_but, the truth is_

**1. when I use "," tokenizer and synonyms**

```auto
PUT test_index
{
    "settings": {
        "index": {
            "number_of_shards": 5,
            "number_of_replicas": 1,
            "analysis": {
                "analyzer": {   
                    "index_analyzer": {
                        "tokenizer": "standard",
                        "filter": ["lowercase"],
                        "type": "custom"
                    },
                    "my_analyzer": {
                        "tokenizer": "comma",
                        "filter": ["my_synonym","lowercase"],
                        "type": "custom"
                    }
                },
                "filter": {
                    "my_synonym": {
                        "ignore_case": "true",
                        "expand": "true",
                        "type": "synonym",
                        "synonyms_path": "analysis/synonym.txt"
                    }
                },
                "tokenizer":{
					"comma":{
						"type": "pattern",
						"pattern":",|，"
					}
			    }
            }
        }
    },
    "mappings": {
        "properties": {
            "abstract": {
                "type": "text",
                "analyzer": "index_analyzer",
                "search_analyzer": "my_analyzer"
            }
        }
    }
}

GET test_index/_analyze
{
   "analyzer":"my_analyzer",
   "text": "thiopental allergy is a test"
}

```

result:

```auto
{
  "tokens": [
    {
      "token": "thiopental allergy is a test",
      "start_offset": 0,
      "end_offset": 28,
      "type": "word",
      "position": 0
    }
  ]
}

```

**2. when I use standard tokenizer and synonyms**

```auto
PUT test_index2
{
    "settings": {
        "index": {
            "number_of_shards": 5,
            "number_of_replicas": 1,
            "analysis": {
                "analyzer": {   
                    "index_analyzer": {
                        "tokenizer": "standard",
                        "filter": ["lowercase"],
                        "type": "custom"
                    },
                    "my_analyzer": {
                        "tokenizer": "standard",
                        "filter": ["my_synonym","lowercase"],
                        "type": "custom"
                    }
                },
                "filter": {
                    "my_synonym": {
                        "ignore_case": "true",
                        "expand": "true",
                        "type": "synonym",
                        "synonyms_path": "analysis/synonymOld.txt"
                    }
                }
            }
        }
    },
    "mappings": {
        "properties": {
            "abstract": {
                "type": "text",
                "analyzer": "index_analyzer",
                "search_analyzer": "my_analyzer"
            }
        }
    }
}

GET test_index2/_analyze
{
   "analyzer":"my_analyzer",
   "text": "thiopental allergy is a test"
}

```

result

```auto
{
  "tokens": [
    {
      "token": "thiopental",
      "start_offset": 0,
      "end_offset": 10,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "penthiobarbital",
      "start_offset": 0,
      "end_offset": 10,
      "type": "SYNONYM",
      "position": 0
    },
    {
      "token": "pentothiobarbital",
      "start_offset": 0,
      "end_offset": 10,
      "type": "SYNONYM",
      "position": 0
    },
    {
      "token": "allergy",
      "start_offset": 11,
      "end_offset": 18,
      "type": "<ALPHANUM>",
      "position": 1
    },
    {
      "token": "allergy",
      "start_offset": 11,
      "end_offset": 18,
      "type": "SYNONYM",
      "position": 1
    },
    {
      "token": "allergy",
      "start_offset": 11,
      "end_offset": 18,
      "type": "SYNONYM",
      "position": 1
    },
    {
      "token": "is",
      "start_offset": 19,
      "end_offset": 21,
      "type": "<ALPHANUM>",
      "position": 2
    },
    {
      "token": "a",
      "start_offset": 22,
      "end_offset": 23,
      "type": "<ALPHANUM>",
      "position": 3
    },
    {
      "token": "test",
      "start_offset": 24,
      "end_offset": 28,
      "type": "<ALPHANUM>",
      "position": 4
    }
  ]
}

```

Neither of these results met my needs

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [August 15, 2023, 11:01am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/7 "2023-08-15T11:01:34Z")

</div>

May be adding [shingles](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-shingle-tokenfilter.html) before the synonyms token filter would help?

---

<div class="post-metadata">

### Author: ![1057888035](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/1057888035/32/124586_2.png) [@1057888035](https://discuss.elastic.co/u/1057888035)
#### Post date: [August 17, 2023, 1:52am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/8 "2023-08-17T01:52:57Z")

</div>

result this error: **Token filter [shingle] cannot be used to parse synonyms**

```auto
PUT test_index
{
    "settings": {
        "index": {
            "number_of_shards": 5,
            "number_of_replicas": 1,
            "analysis": {
                "analyzer": {   
                    "index_analyzer": {
                        "tokenizer": "standard",
                        "type": "custom"
                    },
                    "my_analyzer": {
                        "tokenizer": "whitespace",
                        "filter": ["shingle","my_synonym"],
                        "type": "custom"
                    }
                },
                "filter": {
                    "my_synonym": {
                        "ignore_case": "true",
                        "expand": "true",
                        "type": "synonym",
                        "synonyms_path": "analysis/synonym.txt"
                    }
                }
            }
        }
    },
    "mappings": {
        "properties": {
            "abstract": {
                "type": "text",
                "analyzer": "index_analyzer",
                "search_analyzer": "my_analyzer"
            }
        }
    }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 14, 2023, 1:53am UTC](https://discuss.elastic.co/t/index-256-out-of-bounds-for-length-256/340710/9 "2023-09-14T01:53:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
