Term VS. Match | ES 5.0.2


(Chris) #1

Ok, so I just came out of a 2 full day Watcher testing. I have been having issues with the Input body queries with watcher. For example, there are these two links in which one uses "match" in a "must", and another uses "term" in a "match". This is a bit confusing and I had lots of trouble by getting execution_not_needed status form my watches while trying different settings in my query body input. I always made sure that the field being searched for and time where accurate and that an execution should happen. However, using different combinations of "term", "must", "match", "filter", inside a "bool" query. Is there any other more in-depth documentation about watcher in x-pack. I would like to contribute my findings since I ran into many issues.


(Alexander Reelsen) #2

Hey,

let's dissect your post, I think there are a few stages of mixing up things in there - I'd like to fix all of your mentioned problems :slight_smile:

First, thanks for testing watcher, happy to get any feedback!

Now, on to your issues. When you refer to these two links, feel free to include them.

I think one of your main problems is not writing watches, but writing correct queries. This is not a watcher feature per se, but just the way you write queries in Elasticsearch. And Elasticsearch has tons of different queries. Those presented in the watcher documentation are just a very small subset, see the Elasticsearch Query DSL.

So, what is the difference between term and match? A term query requires your term to be an exact match in your inverted index (the data structure that is created when you index a document) - a match query is a lot smarter and is doing some sort of analysis before executing the query (usually the same steps that have been done before indexing a document, like lowercasing, splitting, etc). This is how full text search queries work, but it requires you to understand what those queries do, when writing them.

if you need more in-depth documentation about querying, the above links with references to the query are a good start, but I think you will find the best explanations in the The Definitive Guide.

The next problem on your side, the execution_not_needed status. This watch status marks a watch execution, where the condition did not meet. So if you watch checks for the number of hits being greater than 0, this means, that your search actually returned 0 hits - this in turn is very likely when you for example used a term query searching for Solution, however the term is by default lowercased when being indexed and thus not found. Again, using a match query might help here.

So, what is the best way to get up and running here? Before writing any watch, you should take your time and get the query right, ensure that it returns the data you want, and only then proceed with writing the watch itself. The best tool for writing queries is pretty much Kibana in combination with the console - which features URL completion, and even auto completion for queries and aggregations making your life much easier.

Once your query is doing what you want, you can put it inside a watch. However waiting for the watch to execute is pretty tedious as well, so you should take a look at the Execute Watch API for faster debug cycles. This way, you dont need to put the watch, but you can directly execute without storing.

Hope this helps as a start. If you further issues with watcher, feel free to post them here or mail your findings to ne, firstname.lastname@elastic.co

--Alex


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.