Percolating a document that includes range

Elastic Stack Elasticsearch
1

Hi,
I am using ES percolator to save queries, which include ranges. Everything
works great.
I was just told that some of the documents that I will be percolating will
contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a summary)
that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.
}

The query (Query-1) should percolate, since the ranges intersect. I am not
sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Okay seems like if I give a range as I showed below replacing the [1..5]
with [1,5] ES does the intersection correctly and gives me the correctly
results.
I will throw some more tests. But a confirmation from others would be nice
too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating will
contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a summary)
that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.
}

The query (Query-1) should percolate, since the ranges intersect. I am
not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the [1..5]
with [1,5] ES does the intersection correctly and gives me the correctly
results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a summary)
that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.
}

The query (Query-1) should percolate, since the ranges intersect. I am
not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the [1..5]
with [1,5] ES does the intersection correctly and gives me the correctly
results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.
}

The query (Query-1) should percolate, since the ranges intersect. I am
not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi Igor,
Here is what I did to test:
I saved a query:
{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":1,"to":8}}}]}]}}}}}

So here my range is 1-8

And I percolated on:
1)
"doc": {
"beds": [5,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

"doc": {
"beds": [5,7]}

Both of which returned back a match on my saved query,
which leads me to believe that ranges on percolation are supported. Am I
missing something?

Thanks in advance

-A

On Wednesday, April 10, 2013 11:32:54 AM UTC-7, Ali wrote:

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the [1..5]
with [1,5] ES does the intersection correctly and gives me the correctly
results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.
}

The query (Query-1) should percolate, since the ranges intersect. I am
not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Ranges in percolator are supported. I was just concerned that you are using
wrong ranges. Would you expect this query

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":2,"to":5}}}]}]}}}}}

to match this document:

"doc": {
"beds": [1,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

On Wednesday, April 10, 2013 2:55:59 PM UTC-4, Ali wrote:

Hi Igor,
Here is what I did to test:
I saved a query:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":1,"to":8}}}]}]}}}}}

So here my range is 1-8

And I percolated on:
1)
"doc": {
"beds": [5,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

"doc": {
"beds": [5,7]}

Both of which returned back a match on my saved query,
which leads me to believe that ranges on percolation are supported. Am I
missing something?

Thanks in advance

-A

On Wednesday, April 10, 2013 11:32:54 AM UTC-7, Ali wrote:

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the
[1..5] with [1,5] ES does the intersection correctly and gives me the
correctly results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.

}

The query (Query-1) should percolate, since the ranges intersect. I
am not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

I just added some more use cases below:

On Wednesday, April 10, 2013 11:55:59 AM UTC-7, Ali wrote:

Hi Igor,
Here is what I did to test:
I saved a query:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":1,"to":8}}}]}]}}}}}

So here my range is 1-8

And I percolated on:
1)
"doc": {
"beds": [5,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

"doc": {
"beds": [5,7]}

  1. If I percolate on: "doc": { "beds": [0,10]} in this case both my
    min and max are outside the saved query range.
    I do not get a hit. I am not sure why.

  1. If I percolate on: "doc" {"beds": [0,5]} only my min is less than
    query min but max is less than query max.
    I get a hit.

  2. If I percolate on: "doc": {"beds": [2, 10] in this case min is
    greater than saved min but less than saved max, and max is greater than
    saved max. I get a hit.

Both of which returned back a match on my saved query,
which leads me to believe that ranges on percolation are supported. Am I
missing something?

Thanks in advance

-A

On Wednesday, April 10, 2013 11:32:54 AM UTC-7, Ali wrote:

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the
[1..5] with [1,5] ES does the intersection correctly and gives me the
correctly results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a range.

}

The query (Query-1) should percolate, since the ranges intersect. I
am not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

That's exactly what I was trying to explain. [0,5] is not a range. It's
just two values 0 and 5. If ether of them is inside your query range, you
get a hit, if they are outside, you don't. To match ranges you need to
store these two values in different fields. For examples, you can index
range in document

"beds_min": 0,
"beds_max": 5

and implement range in query like this:

{
"and": [{
"range": {
"beds_min": {
"lte": 8,
}
},
"range": {
"beds_max": {
"gte": 1,
}
}
}]
}

On Wednesday, April 10, 2013 3:30:28 PM UTC-4, Ali wrote:

I just added some more use cases below:

On Wednesday, April 10, 2013 11:55:59 AM UTC-7, Ali wrote:

Hi Igor,
Here is what I did to test:
I saved a query:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":1,"to":8}}}]}]}}}}}

So here my range is 1-8

And I percolated on:
1)
"doc": {
"beds": [5,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

"doc": {
"beds": [5,7]}

  1. If I percolate on: "doc": { "beds": [0,10]} in this case both my
    min and max are outside the saved query range.
    I do not get a hit. I am not sure why.

  1. If I percolate on: "doc" {"beds": [0,5]} only my min is less than
    query min but max is less than query max.
    I get a hit.

  2. If I percolate on: "doc": {"beds": [2, 10] in this case min is
    greater than saved min but less than saved max, and max is greater than
    saved max. I get a hit.

Both of which returned back a match on my saved query,
which leads me to believe that ranges on percolation are supported. Am I
missing something?

Thanks in advance

-A

On Wednesday, April 10, 2013 11:32:54 AM UTC-7, Ali wrote:

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the
[1..5] with [1,5] ES does the intersection correctly and gives me the
correctly results.
I will throw some more tests. But a confirmation from others would be
nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be percolating
will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a
range.
}

The query (Query-1) should percolate, since the ranges intersect. I
am not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

Thank you very much Igor, that explains it.

On Wednesday, April 10, 2013 12:40:10 PM UTC-7, Igor Motov wrote:

That's exactly what I was trying to explain. [0,5] is not a range. It's
just two values 0 and 5. If ether of them is inside your query range, you
get a hit, if they are outside, you don't. To match ranges you need to
store these two values in different fields. For examples, you can index
range in document

"beds_min": 0,
"beds_max": 5

and implement range in query like this:

{
"and": [{
"range": {
"beds_min": {
"lte": 8,
}
},
"range": {
"beds_max": {
"gte": 1,
}
}
}]
}

On Wednesday, April 10, 2013 3:30:28 PM UTC-4, Ali wrote:

I just added some more use cases below:

On Wednesday, April 10, 2013 11:55:59 AM UTC-7, Ali wrote:

Hi Igor,
Here is what I did to test:
I saved a query:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"beds":{"from":1,"to":8}}}]}]}}}}}

So here my range is 1-8

And I percolated on:
1)
"doc": {
"beds": [5,7], "baths": 3, "price": [8500, 10000], "photo_count": 11,
"postal_code": "95130"}"

"doc": {
"beds": [5,7]}

  1. If I percolate on: "doc": { "beds": [0,10]} in this case both my
    min and max are outside the saved query range.
    I do not get a hit. I am not sure why.

  1. If I percolate on: "doc" {"beds": [0,5]} only my min is less than
    query min but max is less than query max.
    I get a hit.

  2. If I percolate on: "doc": {"beds": [2, 10] in this case min is
    greater than saved min but less than saved max, and max is greater than
    saved max. I get a hit.

Both of which returned back a match on my saved query,
which leads me to believe that ranges on percolation are supported. Am
I missing something?

Thanks in advance

-A

On Wednesday, April 10, 2013 11:32:54 AM UTC-7, Ali wrote:

Thanks Igor, I did think of enumerating but it won't work for "price"
attributes in the sense that the enumeration will be large.

On Wednesday, April 10, 2013 9:28:26 AM UTC-7, Igor Motov wrote:

What happend in your example is your indexed values 1 and 5 in your
document and the value 1 matched range in query 1 - 4. If your query had
range 2-4 it wouldn't have matched. If you really want to check for
intersecting ranges, the simplest (but not very elegant) solution in your
case would to index [1,2,3,4,5] Because the number of rooms is limited that
might be the best solution for you. Alternatively, you can follow a more
generic approach that is described here -
datetime - Determine Whether Two Date Ranges Overlap - Stack Overflow

On Tuesday, April 9, 2013 5:17:52 PM UTC-4, Ali wrote:

Okay seems like if I give a range as I showed below replacing the
[1..5] with [1,5] ES does the intersection correctly and gives me the
correctly results.
I will throw some more tests. But a confirmation from others would
be nice too.

Thanks,
-A

On Tuesday, April 9, 2013 1:20:38 PM UTC-7, Ali wrote:

Hi,
I am using ES percolator to save queries, which include ranges.
Everything works great.
I was just told that some of the documents that I will be
percolating will contain ranges since these are summary documents.
For example if I have saved a query that looks like:

Query-1:

{"query":{"filtered":{"query":{"match_all":{}},"filter":{"and":[{"and":[{"range":{"baths":{"from":1,"to":5}}},{"range":{"beds":{"from":1,"to":4}}}]}]}}}}}]}}

and later I want to percolate a document (which is essentially a
summary) that is something like:

"doc":{
"baths": 2,
"beds": [1..5] //Note I am making this up to give a
range.
}

The query (Query-1) should percolate, since the ranges intersect. I
am not sure if this is even possible in ES.

I would really appreciate any ideas.

Thanks in advance

-A

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.