Storage problem with ngram filters

Elastic Stack Elasticsearch
1

Hi,

i need to search all sorts of word combinations and sub string of words through my search engine ..
to achieve this i am using ngram filters and it is working perfectly okay but the main concern here is the storage space .. for 23000 documents it is taking upto 1.7 gb space in my local system .. i want to reduce it because the live server have many more records .. so how could i reduce this storage amount ..

anyway to use ngram in a more efficient way ?

Thanks ..

2

Unfortunately, that is the main drawback of ngrams. What are your min and
max gram settings? Can you use an edge ngrams instead? If your max ngram
settings is high, consider reducing it.

3

right now my min and max ngram settings are : 3 - 40

but there are cases where i need longer max values ..

is there any other alternative for ngram?

any other way to match sub strings and all ? like regular expression etc

4

You can use wildcard query but it will be slow.

5

can you elaborate it a bit

what you means by wildcard query ?

and one more thing if i have a title column and it contain a phrase or a sentence so to search for that phrase or sentence is ngram the best option or there is some other way possible to do proper search for sentences and phrases ?

6

I meant that: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/query-dsl-wildcard-query.html

7

i read about this .. but it will affect the search time later on ..

and can you tell me the best way to search for phrases in elasticsearch ?

8

but it will affect the search time later on ..

Yes. You always have a price to pay:

  • indexing throughput and disk size (you pay at index time)
  • search response time (you pay at search time)

No miracle here I'm afraid.

and can you tell me the best way to search for phrases in elasticsearch ?

That's another question. Better to open another question.
But here is an idea: Match phrase query | Elasticsearch Guide [8.11] | Elastic

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.