Search Issue

Elastic Stack Elasticsearch
1

Hi Team,

I have used the following mapping properties for a field called field1 :

"field1" : {
"type" : "string",
"omit_norms" : true,
"omit_term_freq_and_positions" : true
}

Then injected some docs which contains values like "app value1" and "app-value1" for the app field.

Now when I enter query strings like following I am not getting back any data -

  1. "query" : {
    "query_string" : {
    "query" : "+field1:"app value1"",
    "default_operator" : "and",
    "analyzer" : "search_analyzer",
    "allow_leading_wildcard" : false,
    "analyze_wildcard" : true
    }
    }

  2. "query" : {
    "query_string" : {
    "query" : "+field1:"app-value1"",
    "default_operator" : "and",
    "analyzer" : "search_analyzer",
    "allow_leading_wildcard" : false,
    "analyze_wildcard" : true
    }
    }

When I enter query string like below I get back the data-

"query" : {
"query_string" : {
"query" : "+field1:"app"",
"default_operator" : "and",
"analyzer" : "search_analyzer",
"allow_leading_wildcard" : false,
"analyze_wildcard" : true
}
}

It means when I am looking for entire field value like "app value1" or app-value1 no data is retrieved, but when I search for fractions like app or value1 data is returned.

Please provide inputs on it. How will I be able to get the results when I search for whole value and not for fractions only.

Thanks..

2

Hiya

  1. "query" : {
    "query_string" : {
    "query" : "+field1:"app value1"",
    "default_operator" : "and",
    "analyzer" : "search_analyzer",
    "allow_leading_wildcard" : false,
    "analyze_wildcard" : true
    }
    }

You are doing a phrase search ie docs with 'app' followed directly by
'value1'. But you've specified "omit_term_freq_and_positions". So it
doesn't know about the order of your terms.

Please provide inputs on it. How will I be able to get the results when I
search for whole value and not for fractions only.

It sounds like you want this field to do exact matching, rather than
full text queries.

So set it to {index: "not_analyzed"} and search for it with a "term"
query or filter.

This may or may not be the right answer, depending on what this data is
and how you want to query it.

clint

3

Hi Clinton,

Now I have changed the mapping property like below :

"field1" : {"type" : "string","omit_norms" : true, "omit_term_freq_and_positions" : true,"search_analyzer" : "keyword","index_analyzer" : "keyword"}

I am getting the results back for whole values but fractional searches are failing now.

I have also tried index: "not_analyzed" in mapping and whole value searches worked but not fractional.

Does it make sense to use "search_analyzer" : "keyword" and "index_analyzer" : "keyword" in combination with "omit_norms" : true and "omit_term_freq_and_positions" : true.

Thanks..

4

Hiya

Now I have changed the mapping property like below :

"field1" : {"type" : "string","omit_norms" : true,
"omit_term_freq_and_positions" : true,"search_analyzer" :
"keyword","index_analyzer" : "keyword"}

I am getting the results back for whole values but fractional searches are
failing now.

OK - that's what i meant by "this may or may not be the right answer".
So you want to do full text searches, but allow for phrase searches.

In which case, keep the mapping as it was, but remove the
"omit_term_freq_and_positions"

That will allow your phrase (ie positional) searches to work.

clint

5

Any input / suggestions on this !

6

Hi Clinton,

Thanks for the reply. If you could give us insight into what impact we will have on performance and memory optimization if we set omit_term_freq_and_positions to false or remove it completely..

Thanks..