Range Filter Aggregation on array

Elastic Stack Elasticsearch
1

Hi all,

I have some troubles when I try to use Range Filter aggregation on an array.

Example of 1 document :

{
   "_index": "test",
   "_type": "values",
   "_id": "1",
   "_version": 1,
   "found": true,
   "_source": {
      "array": [927,425,455,120]
    }
}

For all my "values" document, I'd like to count for "array" field how many
numbers are less than 200 and how many are greater than 500.

I tried this aggregation :

GET /test/values/_search

{
  "aggs" : {
    "less" : {
      "filter":{"range":{"array":{ "lt" : 200}}}
    },
    "greater" : {
       "filter":{"range":{"array":{ "gt" : 500}}}
    }
  }
}

But the 1st filter count the number of documents which have an array
containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500
(not how many document).

If I make a sum / min / max aggregation, it will be on each value in arrays
but not with a filter.
Do you have an idea how to do that thing?

I did it with 2 script filters, it works, but the computing time is too bad
:

{ 
  "aggs" : {
    "less" : {
      "sum":{
        "script":" def sum = 0; doc['array'].values.each(){if(it < 200) 
sum++}; return sum;"}
    },
    "greater" : {
       "sum":{
         "script":"def sum = 0; doc['array'].values.each(){if(it > 500) 
sum++}; return sum;"}
    }
}

Any Idea?
Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

One of the solution I can think is by using nested documents for your array.
In that case, each element of the array will be seen as the document.

Hope this helps

--
David Pilato | Technical Advocate | elasticsearch.com
david.pilato@elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi.nonnon@gmail.com) a écrit:

Hi all,

I have some troubles when I try to use Range Filter aggregation on an array.

Example of 1 document :

{
   "_index": "test",
   "_type": "values",
   "_id": "1",
   "_version": 1,
   "found": true,
   "_source": {
      "array": [927,425,455,120]
    }
}

For all my "values" document, I'd like to count for "array" field how many numbers are less than 200 and how many are greater than 500.

I tried this aggregation :

GET /test/values/_search

{
  "aggs" : {
    "less" : {
      "filter":{"range":{"array":{ "lt" : 200}}}
    },
    "greater" : {
       "filter":{"range":{"array":{ "gt" : 500}}}
    }
  }
}

But the 1st filter count the number of documents which have an array containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500 (not how many document).

If I make a sum / min / max aggregation, it will be on each value in arrays but not with a filter.
Do you have an idea how to do that thing?

I did it with 2 script filters, it works, but the computing time is too bad :

{ 
  "aggs" : {
    "less" : {
      "sum":{
        "script":" def sum = 0; doc['array'].values.each(){if(it < 200) sum++}; return sum;"}
    },
    "greater" : {
       "sum":{
         "script":"def sum = 0; doc['array'].values.each(){if(it > 500) sum++}; return sum;"}
    }
}

Any Idea?
Thanks!

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5433a689.836c40e.7bd4%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

3

Hi,

Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted a
new one)

I did it with nested documents but the array seems to be faster on other
aggregations like extended stats or percentiles then I wondered if my
filters are well written.

Thanks again.

Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :

One of the solution I can think is by using nested documents for your
array.
In that case, each element of the array will be seen as the document.

Hope this helps

--
David Pilato | Technical Advocate | elasticsearch.com
http://elasticsearch.com
david....@elasticsearch.com <javascript:>
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs

Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi....@gmail.com
<javascript:>) a écrit:

Hi all,

I have some troubles when I try to use Range Filter aggregation on an
array.

Example of 1 document :

{
  "_index": "test",
  "_type": "values",
  "_id": "1",
  "_version": 1,
  "found": true,
  "_source": {
     "array": [927,425,455,120]
   }
}

For all my "values" document, I'd like to count for "array" field how many
numbers are less than 200 and how many are greater than 500.

I tried this aggregation :

GET /test/values/_search

{
  "aggs" : {
    "less" : {
      "filter":{"range":{"array":{ "lt" : 200}}}
    },
    "greater" : {
       "filter":{"range":{"array":{ "gt" : 500}}}
    }
  }
}

But the 1st filter count the number of documents which have an array
containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500
(not how many document).

If I make a sum / min / max aggregation, it will be on each value in
arrays but not with a filter.
Do you have an idea how to do that thing?

I did it with 2 script filters, it works, but the computing time is too
bad :

 { 
  "aggs" : {
    "less" : {
      "sum":{
        "script":" def sum = 0; doc['array'].values.each(){if(it < 200) 
sum++}; return sum;"}
    },
    "greater" : {
       "sum":{
         "script":"def sum = 0; doc['array'].values.each(){if(it > 500) 
sum++}; return sum;"}
    }
 }

Any Idea?
Thanks!

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

I have no idea. It's hard to understand what you exactly did without a full reproduction.
A SENSE script posted as a GIST would help a lot I think.

--
David Pilato | Technical Advocate | elasticsearch.com
david.pilato@elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 7 octobre 2014 à 10:49:54, Rémi Nonnon (remi.nonnon@gmail.com) a écrit:

Hi,

Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted a new one)

I did it with nested documents but the array seems to be faster on other aggregations like extended stats or percentiles then I wondered if my filters are well written.

Thanks again.

Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :
One of the solution I can think is by using nested documents for your array.
In that case, each element of the array will be seen as the document.

Hope this helps

--
David Pilato | Technical Advocate | elasticsearch.com
david....@elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi....@gmail.com) a écrit:

Hi all,

I have some troubles when I try to use Range Filter aggregation on an array.

Example of 1 document :

{
   "_index": "test",
   "_type": "values",
   "_id": "1",
   "_version": 1,
   "found": true,
   "_source": {
      "array": [927,425,455,120]
    }
}

For all my "values" document, I'd like to count for "array" field how many numbers are less than 200 and how many are greater than 500.

I tried this aggregation :

GET /test/values/_search

{
  "aggs" : {
    "less" : {
      "filter":{"range":{"array":{ "lt" : 200}}}
    },
    "greater" : {
       "filter":{"range":{"array":{ "gt" : 500}}}
    }
  }
}

But the 1st filter count the number of documents which have an array containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500 (not how many document).

If I make a sum / min / max aggregation, it will be on each value in arrays but not with a filter.
Do you have an idea how to do that thing?

I did it with 2 script filters, it works, but the computing time is too bad :

{ 
  "aggs" : {
    "less" : {
      "sum":{
        "script":" def sum = 0; doc['array'].values.each(){if(it < 200) sum++}; return sum;"}
    },
    "greater" : {
       "sum":{
         "script":"def sum = 0; doc['array'].values.each(){if(it > 500) sum++}; return sum;"}
    }
}

Any Idea?
Thanks!

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5433b483.614fd4a1.7bd4%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

5

Hi,

It's ok, I think you are right, nested will be the solution.

Regards,
Rémi.

Le mardi 7 octobre 2014 11:38:44 UTC+2, David Pilato a écrit :

I have no idea. It's hard to understand what you exactly did without a
full reproduction.
A SENSE script posted as a GIST would help a lot I think.

--
David Pilato | Technical Advocate | elasticsearch.com
http://elasticsearch.com
david....@elasticsearch.com <javascript:>
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs

Le 7 octobre 2014 à 10:49:54, Rémi Nonnon (remi....@gmail.com
<javascript:>) a écrit:

Hi,

Thanks for your immediate help!
(sorry i deleted this post because I didn't find how to edit it. I posted
a new one)

I did it with nested documents but the array seems to be faster on other
aggregations like extended stats or percentiles then I wondered if my
filters are well written.

Thanks again.

Le mardi 7 octobre 2014 10:38:43 UTC+2, David Pilato a écrit :

One of the solution I can think is by using nested documents for your
array.
In that case, each element of the array will be seen as the document.

Hope this helps

--

Le 7 octobre 2014 à 10:36:28, Rémi Nonnon (remi....@gmail.com) a écrit:

Hi all,

I have some troubles when I try to use Range Filter aggregation on an
array.

Example of 1 document :

{
  "_index": "test",
  "_type": "values",
  "_id": "1",
  "_version": 1,
  "found": true,
  "_source": {
     "array": [927,425,455,120]
   }
}

For all my "values" document, I'd like to count for "array" field how
many numbers are less than 200 and how many are greater than 500.

I tried this aggregation :

GET /test/values/_search

{
  "aggs" : {
    "less" : {
      "filter":{"range":{"array":{ "lt" : 200}}}
    },
    "greater" : {
       "filter":{"range":{"array":{ "gt" : 500}}}
    }
  }
}

But the 1st filter count the number of documents which have an array
containing a value <200 and the 2nd how many have a value >500.
What I'd like is to count how many values are <200 and how many are >500
(not how many document).

If I make a sum / min / max aggregation, it will be on each value in
arrays but not with a filter.
Do you have an idea how to do that thing?

I did it with 2 script filters, it works, but the computing time is too
bad :

 { 
  "aggs" : {
    "less" : {
      "sum":{
        "script":" def sum = 0; doc['array'].values.each(){if(it < 200) 
sum++}; return sum;"}
    },
    "greater" : {
       "sum":{
         "script":"def sum = 0; doc['array'].values.each(){if(it > 500) 
sum++}; return sum;"}
    }
 }

Any Idea?
Thanks!

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d988da81-167f-48d2-b4c8-8e94956bc333%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ea6b7b2e-2874-4387-b314-86a425e88b77%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/534e9e53-32af-4660-b56c-e63c01006b21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.