# Exact match now works but why is my not\_analyzed field split into tokens?

**URL:** <https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798>\
**Category:** Elasticsearch\
**Created:** [November 28, 2015, 9:32am UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798 "2015-11-28T09:32:31Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![setuk](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@setuk](https://discuss.elastic.co/u/setuk)\
**Post date:** [November 28, 2015, 9:32am UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/1 "2015-11-28T09:32:31Z")

</div>

Continuing the discussion from [Exact match not working](https://discuss.elastic.co/t/exact-match-not-working/35787/6):

> [@Exact match not working](https://discuss.elastic.co/t/exact-match-not-working/35787/6):
>
> > [@Exact match not working](https://discuss.elastic.co/t/exact-match-not-working/35787/5):
> >
> > curl -XPOST [http://192.168.134.179:9200/testnames2/test](http://192.168.134.179:9200/testnames2/test)
> 
> Your POST is to the wrong resource. You need to POST to your index (i.e. testnames2). Your type hrname and its mapping was never create. Notice the type field of the response.
> 
> > [@Exact match not working](https://discuss.elastic.co/t/exact-match-not-working/35787/5):
> >
> > Create the index with a single type, and one field which is not\_analyzed  
> > {"\_index":"testnames2","\_type":"test","\_id":"AVFKeAndR0bg8LfMuKAG","\_version":1,"\_shards":{"total":2,"successful":1,"failed":0},"created":true}

```auto
printf "\nDelete the index to start from scratch\n";
curl -XDELETE 'http://192.168.134.179:9200/testnames2'
printf "\nCreate the index with a single type, and one field which is not_analyzed\n";
curl -XPOST http://192.168.134.179:9200/testnames2 -d '{
    "settings" : {
        "number_of_shards" : 1
    },
    "mappings" : {
        "hrname" : {
            "_all" : { "enabled" : false },
            "properties" : {
                "rawLookup" : { "type" : "string", "index" : "not_analyzed" }
            }
        }
    }
}'
printf "\nCheck mappings have persisted\n";
curl -XGET 'http://192.168.134.179:9200/testnames2/_mappings/?pretty=true'

printf "\nAdd one record into the index of the right type\n";
curl -XPUT 'http://192.168.134.179:9200/testnames2/hrname/1' -d '{
    "rawLookup" : "Simon Taylor"
}'
printf "\nRefresh index\n";
curl -XPOST 'http://192.168.134.179:9200/testnames2/_refresh'

printf "\nAttempt to get the field back using a filtered query using one term with the exact text i inputted\n";
curl -XGET 'http://192.168.134.179:9200/testnames2/_search/?pretty=true' -d '
{
      "query" : {
         "filtered" : {
            "filter" : {
               "term" : {
                  "rawLookup" : "Simon Taylor"
               }
            }
         }
      }
}
'
printf "Now just using Query Term RawLookup\n";
curl -XGET 'http://192.168.134.179:9200/testnames2/_search/?pretty=true' -d '
{
      "query" : {
               "term" : {
                  "rawLookup" : "Simon Taylor"
                }
        }
}
'
printf "Output of analyze"
curl -XGET http://192.168.134.179:9200/testnames2/_analyze?pretty=1,field=rawLookup -d'
Simon Taylor'

Generates this:-

Delete the index to start from scratch
{"acknowledged":true}
Create the index with a single type, and one field which is not_analyzed
{"acknowledged":true}
Check mappings have persisted
{
  "testnames2" : {
    "mappings" : {
      "hrname" : {
        "_all" : {
          "enabled" : false
        },
        "properties" : {
          "rawLookup" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      }
    }
  }
}

Add one record into the index of the right type
{"_index":"testnames2","_type":"hrname","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}
Refresh index
{"_shards":{"total":2,"successful":1,"failed":0}}
Attempt to get the field back using a filtered query using one term with the exact text i inputted
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "testnames2",
      "_type" : "hrname",
      "_id" : "1",
      "_score" : 1.0,
      "_source":{
    "rawLookup" : "Simon Taylor"
}
    } ]
  }
}
Now just using Query Term RawLookup
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.30685282,
    "hits" : [ {
      "_index" : "testnames2",
      "_type" : "hrname",
      "_id" : "1",
      "_score" : 0.30685282,
      "_source":{
    "rawLookup" : "Simon Taylor"
}
    } ]
  }
}
Output of analyze{
  "tokens" : [ {
    "token" : "simon",
    "start_offset" : 1,
    "end_offset" : 6,
    "type" : "<ALPHANUM>",
    "position" : 0
  }, {
    "token" : "taylor",
    "start_offset" : 7,
    "end_offset" : 13,
    "type" : "<ALPHANUM>",
    "position" : 1
  } ]
}
Simons-MacBook-Pro:elastic simon$ 

```

---

<div class="post-metadata">

**Author:** ![setuk](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@setuk](https://discuss.elastic.co/u/setuk)\
**Post date:** [November 28, 2015, 3:05pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/2 "2015-11-28T15:05:25Z")

</div>

What should i expect the output of analyze to look like for a not\_analyzed field?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 28, 2015, 3:25pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/3 "2015-11-28T15:25:57Z")

</div>

I guess it should be something like:

```auto
$ curl "localhost:9200/_analyze?pretty&analyzer=keyword" -d "Simon Taylor"
{
  "tokens" : [ {
    "token" : "Simon Taylor",
    "start_offset" : 0,
    "end_offset" : 12,
    "type" : "word",
    "position" : 0
  } ]
}

```

---

<div class="post-metadata">

**Author:** ![setuk](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@setuk](https://discuss.elastic.co/u/setuk)\
**Post date:** [November 28, 2015, 3:51pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/4 "2015-11-28T15:51:43Z")

</div>

Thanks,  
I note your version pays no reference to the index or the field I am interested in looking at.  
Is the way to interpret what you have done is - take some text and run it through an analyzer and see how it tokenizes the text.  
This may bear no relation to how a specific mapping for a specific index has tokenized data that is already there.

Is there a way to test how something has already been analyzed or not analyzed (as the case may be)?

I was hoping that something like the following would do it:-

```auto
printf "Output of analyze"
curl -XGET "http://192.168.134.179:9200/testnames2/_analyze?pretty=1&analyzer=keyword&field=rawLookup" -d "Simon Taylor"

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 28, 2015, 5:20pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/5 "2015-11-28T17:20:15Z")

</div>

Can you try with a json request? May be fields is not supported as a parameter?

See [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html)

---

<div class="post-metadata">

**Author:** ![setuk](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@setuk](https://discuss.elastic.co/u/setuk)\
**Post date:** [November 28, 2015, 5:23pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/6 "2015-11-28T17:23:12Z")

</div>

i found this syntax which does the job:-

```auto
printf "Output of analyze"
curl -XGET "http://192.168.134.179:9200/testnames2/_analyze?pretty=1" -d '
{
    "field" : "rawLookup",
    "text" : "Simon Taylor"
}
'

```

---

<div class="post-metadata">

**Author:** ![vtst2412](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vtst2412/32/6228_2.png) [@vtst2412](https://discuss.elastic.co/u/vtst2412)\
**Post date:** [November 28, 2015, 8:57pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/7 "2015-11-28T20:57:34Z")

</div>

That is working for the wrong reason. You are not actually testing whether the text "Simon Taylor" in your **rawLookup** field has been analyzed. Technically, the \_analyze method here is using the analyzer used to analyze the field **rawLookup** of this index to analyze any text you're inputting into the parameter (which you happened to pick to be "Simon Taylor" in this case. You can put "I love Elasticsearch" for all it cares).

Overall, I think the test still does an approximation of what you're trying to determine: whether the field **rawLookup** on index **testname2** is "not\_analyzed" (we know the field is "not\_analyzed" because its mapping is using a **keyword analyzer** to analyze the text you provide it in the parameter of the request).

Cheers.

---

<div class="post-metadata">

**Author:** ![setuk](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@setuk](https://discuss.elastic.co/u/setuk)\
**Post date:** [November 29, 2015, 4:18pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/8 "2015-11-29T16:18:11Z")

</div>

ok makes sense.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:35pm UTC](https://discuss.elastic.co/t/exact-match-now-works-but-why-is-my-not-analyzed-field-split-into-tokens/35798/9 "2017-07-05T23:35:03Z")

</div>


