Inverted index per field


(Ran) #1

Hello,

As a Newbie I wonder how the transition is made from the inverted index to a field.
In all documentation I see about ES inverted index shows terms -> docs

So if my doc contain this two fields -
1 with very long text in english - Enligh analyzer
2. contain string like user name - ngram analyzer as I want to enable partial matching here

Now some one use a prefix filter on the text field, will it go over the terms generated from user name field ?
I would expect behind the scene to have inverted index per field but I didn't see it anywhere...

Ran.


(Doug Turnbull) #2

Each field is its own self contained inverted index and searching universe. The best way to see that is to dig into the organization of Lucene data structures. I agree its one of those points that's not stated well, but trust me that's how its organized.

The trick with search is not just getting one field's matching right, but searching across all the fields and balancing their scores. As fields have their own statistics related to their inverted index, this can be rather challenging. Each field is its own little weird search scoring universe. We write a lot about multifield search in our book, but you can also just read the ES definitive guide section on multifield search which introduces the ideas.


(system) #3