Search for specific symbols like %


(Arthur Kushman) #1

We are using an ES in our project and can not find the answer or an example, how to search for ints with percent character, ex.:
we place in query match "50%" ES found the 50 and 50%, as we thought it would be a special character for ES and tried out all the analyzers like whitespace, keyword, escaping character % - nothing helped.

Also, the 2nd problem is that the highlighter doesn't enclose the whole expression into tags, ex.: <span class="highlight">50</span>%, but we expect - <span class="highlight">50%</span>

Hope somebody have the exact same experience and help, thx.


(Abdon Pijpelink) #2

Not sure why it doesn't work for you, but I suspect it is an analysis issue. The below works for me, applying the whitespace analyzer. A query for just "50" returns no hits, but a query for "50%" does, with the correct highlighting: "foo <em>50%</em> bar"

PUT test
{
  "mappings": {
    "doc": {
      "properties": {
        "my_field": {
          "type": "text", 
          "analyzer": "whitespace"
        }
      }
    }
  }
}

PUT test/doc/1
{
  "my_field": "foo 50% bar"
}

# Returns nothing
GET test/_search
{
  "query": {
    "match": {
      "my_field": "50"
    }
  }
}

# Returns the correct doc with correct highlighting
GET test/_search
{
  "query": {
    "match": {
      "my_field": "50%"
    }
  },
  "highlight": {
    "fields": {
      "my_field": {}
    }
  }
}

(Arthur Kushman) #3

We have an existing index and try to update like this:
curl -X PUT 'http://localhost:9200/spir/material/_mapping?ignore_conflicts=true' -d
'{
"material": {
"properties": {
"text": {
"type": "text",
"analyzer": "whitespace"
},
"headline": {
"type": "text", "analyzer": "whitespace"
},
"lead": {
"type": "text", "analyzer": "whitespace"
}
}
}
}'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"}],"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"},"status":400}%

Then we've tried to update without ignore_conflicts=true getting another error about conflicts:
curl -X PUT 'http://localhost:9200/spir/material/_mapping?ignore_conflicts=true' -d
'{
"material": {
"properties": {
"text": {
"type": "text",
"analyzer": "whitespace"
},
"headline": {
"type": "text", "analyzer": "whitespace"
},
"lead": {
"type": "text", "analyzer": "whitespace"
}
}
}
}'
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"}],"type":"illegal_argument_exception","reason":"request [/spir/material/_mapping] contains unrecognized parameter: [ignore_conflicts]"},"status":400}

But there are no types other then material:
curl -XGET 'localhost:9200/spir?pretty'
{
"spir" : {
"aliases" : { },
"mappings" : {
"material" : {
"properties" : {
"author_string" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"genre_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"headline" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"keywords" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lang" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"lead" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"location" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"news_date" : {
"type" : "long"
},
"priority" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"products" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"rubric_id" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"service_info" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"slug" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"text" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1503043085847",
"number_of_shards" : "5",
"number_of_replicas" : "1",
"uuid" : "uVQ-lzwqSNOpH2-AhM8tvg",
"version" : {
"created" : "5040299"
},
"provided_name" : "spir"
}
}
}
}


(Abdon Pijpelink) #4

You can't dynamically change the analyzer for an existing field on an existing index. You can only define the analyzer when you create an index (or when you add a new field to an index).


(Arthur Kushman) #5

Relatively to this article I can - https://gist.github.com/nicolashery/6317643

"To update the mappings of this existing index, you need to do it for each type (here we only have the weapons type):"

$ curl -X PUT 'http://localhost:9200/thegame/weapons/_mapping?ignore_conflicts=true' -d
'{
"weapons": {
"properties": {
"name": {
"type": "string",
"analyzer": "full_name"
},
"description": {
"type": "string"
},
"category": {
"type": "string"
}
}
}
}'


(Abdon Pijpelink) #6

That guide is for version 0.9. That is a really old version. Not sure what was possible back then, but nowadays it's definitely not possible to change the mapping of existing fields dynamically.

I'd recommend that you check out the official documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings


(Arthur Kushman) #7

Back then was better, thx.
Hate ES documentation it is like waterfall of needless words.


(Arthur Kushman) #8

Found docs in ES5.5 about possible updates to fields:
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings

It is not about type though.


(Abdon Pijpelink) #9

What you need to do if you want to change the mapping of existing documents is reindex those documents to another index with the correct mapping.

So, first create a new index which you create with the new mapping. Then use the reindex API to get all documents from the old index A into the new index B. As those documents get reindexed, they will get an updated mapping applied to them. More info about the reindex API here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.