# Accessing all properties of an object field **URL:** \ **Category:** Elasticsearch\ **Created:** [July 5, 2011, 9:02pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782 "2011-07-05T21:02:52Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Daniel\_E](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_e/32/3137_2.png) [@Daniel\_E](https://discuss.elastic.co/u/Daniel_E)\ **Post date:** [July 5, 2011, 9:02pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/1 "2011-07-05T21:02:52Z")
Given documents with a structure like this: {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} How could I use a script (such as in a facet query) to get at all the attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![Daniel\_E](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_e/32/3137_2.png) [@Daniel\_E](https://discuss.elastic.co/u/Daniel_E)\ **Post date:** [July 5, 2011, 11:52pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/2 "2011-07-05T23:52:30Z")
As a follow-up, I've been trying to use Elasticsearch Head's transformation function to aggregate the results, but the problem there is that I have to get the complete result set which exhausts memory in ES. So, I've been trying to come up with a script\_fields statement that would transform the values object into a format that would allow me to facet on terms and then have a single object returned, but I can't find a way to do even that. here is the closest I've come which fails with an error that java.util.HashMap is not iterable: "script\_fields": { "early\_gluestartup\_read\_ops": { "script": "var arr = ; for (key : _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; arr.reduce(function(comb,key){return comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); return comb;" } } On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > Given documents with a structure like this: > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > How could I use a script (such as in a facet query) to get at all the > attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 6, 2011, 1:27am UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/3 "2011-07-06T01:27:39Z")
Maybe you can explain what you are trying to do exactly? I did not understand... On Wednesday, July 6, 2011 at 2:52 AM, Daniel Einspanjer wrote: > As a follow-up, I've been trying to use Elasticsearch Head's transformation function to aggregate the results, but the problem there is that I have to get the complete result set which exhausts memory in ES. > So, I've been trying to come up with a script\_fields statement that would transform the values object into a format that would allow me to facet on terms and then have a single object returned, but I can't find a way to do even that. here is the closest I've come which fails with an error that java.util.HashMap is not iterable: > > "script\_fields": { > "early\_gluestartup\_read\_ops": { > "script": "var arr = ; for (key : _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; arr.reduce(function(comb,key){return comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); return comb;" > } > } > > On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer \<[deinspanjer@gmail.com](mailto:deinspanjer@gmail.com) ([mailto:deinspanjer@gmail.com](mailto:deinspanjer@gmail.com))\> wrote: > > > Given documents with a structure like this: > > > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type > > ":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > > > How could I use a script (such as in a facet query) to get at all the attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![Daniel\_E](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_e/32/3137_2.png) [@Daniel\_E](https://discuss.elastic.co/u/Daniel_E)\ **Post date:** [July 6, 2011, 3:06pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/4 "2011-07-06T15:06:31Z")
Ultimately, I was hoping to get a facet histogram of the histogram data stored in these documents. for instance, in each document, there is an EARLY\_GLUESTARTUP\_READ\_OPS.values object which has properties consisting of the histogram data, key is the histogram bucket and value is the count of entries for that key. It seems to be impossible to facet this data in any useful way, even with the use of script\_fields because I can't enumerate all the property keys under the .values object since I don't know them ahead of time due to their being a histogram. As an example, if I had the following three documents: {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1}}} {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":2,"10":9,"11":0},"sum":1}}} {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":1,"5":11,"11":0},"sum":1}}} I would like to have some efficient way to generate the following aggregate result: {"sum\_histogram":{"0":0,"1":1,"3":3,"5":11,"10":9,"11":0}} But depending on the corpus of documents, I don't know what values will actually be enumerated. -Daniel On Tue, Jul 5, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Maybe you can explain what you are trying to do exactly? I did not > understand... > > On Wednesday, July 6, 2011 at 2:52 AM, Daniel Einspanjer wrote: > > As a follow-up, I've been trying to use Elasticsearch Head's transformation > function to aggregate the results, but the problem there is that I have to > get the complete result set which exhausts memory in ES. > So, I've been trying to come up with a script\_fields statement that would > transform the values object into a format that would allow me to facet on > terms and then have a single object returned, but I can't find a way to do > even that. here is the closest I've come which fails with an error that > java.util.HashMap is not iterable: > > "script\_fields": { > "early\_gluestartup\_read\_ops": { > "script": "var arr = ; for (key : > _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; > arr.reduce(function(comb,key){return > comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); > return comb;" > } > } > > On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > > Given documents with a structure like this: > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > How could I use a script (such as in a facet query) to get at all the > attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![Daniel\_E](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_e/32/3137_2.png) [@Daniel\_E](https://discuss.elastic.co/u/Daniel_E)\ **Post date:** [July 6, 2011, 4:56pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/5 "2011-07-06T16:56:07Z")
I managed to make some progress.. I was able to create a script\_field that expands out all the data in each document's histogram: { "script\_fields": { "early\_gluestartup\_read\_ops": { "script": "var arr = ; var start = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[0]; var end = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[1]; var vals = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values; for (i = start; i \<= end; i++) { if (vals.containsKey(String.valueOf(i))) { for (j = 0; j \< vals[String.valueOf(i)]; j++) { arr.add(i); } } }; return arr;" } }, "query": { "term": { "OS": "winnt" } }, "facets": { "early\_gluestartup\_read\_ops": { "terms": { "field": "early\_gluestartup\_read\_ops" } } } } But I can't facet on the script field. I expect this is the same problem that was just posted in the mailing list where the person wanted to use a script\_field in a term\_stats. Think I've hit a dead end here. Gonna have to do these aggregations in some other system instead. -Daniel On Wed, Jul 6, 2011 at 11:06 AM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > Ultimately, I was hoping to get a facet histogram of the histogram data > stored in these documents. for instance, in each document, there is an > EARLY\_GLUESTARTUP\_READ\_OPS.values object which has properties consisting of > the histogram data, key is the histogram bucket and value is the count of > entries for that key. > > It seems to be impossible to facet this data in any useful way, even with > the use of script\_fields because I can't enumerate all the property keys > under the .values object since I don't know them ahead of time due to their > being a histogram. > > As an example, if I had the following three documents: > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1}}} > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":2,"10":9,"11":0},"sum":1}}} > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":1,"5":11,"11":0},"sum":1}}} > > I would like to have some efficient way to generate the following aggregate > result: > > {"sum\_histogram":{"0":0,"1":1,"3":3,"5":11,"10":9,"11":0}} > > But depending on the corpus of documents, I don't know what values will > actually be enumerated. > > -Daniel > > On Tue, Jul 5, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > > Maybe you can explain what you are trying to do exactly? I did not > > understand... > > > > On Wednesday, July 6, 2011 at 2:52 AM, Daniel Einspanjer wrote: > > > > As a follow-up, I've been trying to use Elasticsearch Head's > > transformation function to aggregate the results, but the problem there is > > that I have to get the complete result set which exhausts memory in ES. > > So, I've been trying to come up with a script\_fields statement that would > > transform the values object into a format that would allow me to facet on > > terms and then have a single object returned, but I can't find a way to do > > even that. here is the closest I've come which fails with an error that > > java.util.HashMap is not iterable: > > > > "script\_fields": { > > "early\_gluestartup\_read\_ops": { > > "script": "var arr = ; for (key : > > _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; > > arr.reduce(function(comb,key){return > > comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); > > return comb;" > > } > > } > > > > On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > > > > Given documents with a structure like this: > > > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > > > How could I use a script (such as in a facet query) to get at all the > > attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![Daniel\_E](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_e/32/3137_2.png) [@Daniel\_E](https://discuss.elastic.co/u/Daniel_E)\ **Post date:** [July 7, 2011, 6:01pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/6 "2011-07-07T18:01:51Z")
Since script\_fields can't be used in facets, I left that approach and went on to trying to refer to each value explicitly. You can see some of the queries we have successfully made here: [http://etherpad.mozilla.org:9000/ep/pad/view/ro.zIxyOfHFden/rev.259](http://etherpad.mozilla.org:9000/ep/pad/view/ro.zIxyOfHFden/rev.259) It is really unfortunate that faceting can't get at the data the way we want. The only way I've thought of to really get it to work would be to have a flat array instead of a histogram. But that would be very expensive in terms of document size since: 5:5000 would turn into 5000 5 elements in the array. -Daniel On Wed, Jul 6, 2011 at 12:56 PM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > I managed to make some progress.. I was able to create a script\_field that > expands out all the data in each document's histogram: > > { > "script\_fields": { > "early\_gluestartup\_read\_ops": { > "script": "var arr = ; var start = > \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[0]; var end = > \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[1]; var vals = > \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values; for (i = start; i \<= > end; i++) { if (vals.containsKey(String.valueOf(i))) { for (j = 0; j \< > vals[String.valueOf(i)]; j++) { arr.add(i); } } }; return arr;" > } > }, > "query": { > "term": { > "OS": "winnt" > } > }, > "facets": { > "early\_gluestartup\_read\_ops": { > "terms": { > "field": "early\_gluestartup\_read\_ops" > } > } > } > } > > But I can't facet on the script field. I expect this is the same problem > that was just posted in the mailing list where the person wanted to use a > script\_field in a term\_stats. Think I've hit a dead end here. Gonna have > to do these aggregations in some other system instead. > > -Daniel > > On Wed, Jul 6, 2011 at 11:06 AM, Daniel Einspanjer [deinspanjer@gmail.com](mailto:deinspanjer@gmail.com)wrote: > > > Ultimately, I was hoping to get a facet histogram of the histogram data > > stored in these documents. for instance, in each document, there is an > > EARLY\_GLUESTARTUP\_READ\_OPS.values object which has properties consisting of > > the histogram data, key is the histogram bucket and value is the count of > > entries for that key. > > > > It seems to be impossible to facet this data in any useful way, even with > > the use of script\_fields because I can't enumerate all the property keys > > under the .values object since I don't know them ahead of time due to their > > being a histogram. > > > > As an example, if I had the following three documents: > > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1}}} > > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":2,"10":9,"11":0},"sum":1}}} > > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":1,"5":11,"11":0},"sum":1}}} > > > > I would like to have some efficient way to generate the following > > aggregate result: > > > > {"sum\_histogram":{"0":0,"1":1,"3":3,"5":11,"10":9,"11":0}} > > > > But depending on the corpus of documents, I don't know what values will > > actually be enumerated. > > > > -Daniel > > > > On Tue, Jul 5, 2011 at 9:27 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > > > > Maybe you can explain what you are trying to do exactly? I did not > > > understand... > > > > > > On Wednesday, July 6, 2011 at 2:52 AM, Daniel Einspanjer wrote: > > > > > > As a follow-up, I've been trying to use Elasticsearch Head's > > > transformation function to aggregate the results, but the problem there is > > > that I have to get the complete result set which exhausts memory in ES. > > > So, I've been trying to come up with a script\_fields statement that would > > > transform the values object into a format that would allow me to facet on > > > terms and then have a single object returned, but I can't find a way to do > > > even that. here is the closest I've come which fails with an error that > > > java.util.HashMap is not iterable: > > > > > > "script\_fields": { > > > "early\_gluestartup\_read\_ops": { > > > "script": "var arr = ; for (key : > > > _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; > > > arr.reduce(function(comb,key){return > > > comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); > > > return comb;" > > > } > > > } > > > > > > On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer \<[deinspanjer@gmail.com](mailto:deinspanjer@gmail.com) > > > > > > > wrote: > > > > > > Given documents with a structure like this: > > > > > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histogram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > > > > > How could I use a script (such as in a facet query) to get at all the > > > attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 8, 2011, 11:15pm UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/7 "2011-07-08T23:15:42Z")
I am still not sure I fully understand it..., maybe jump on IRC and we can have more realtime discussion? On Thursday, July 7, 2011 at 9:01 PM, Daniel Einspanjer wrote: > Since script\_fields can't be used in facets, I left that approach and went on to trying to refer to each value explicitly. You can see some of the queries we have successfully made here: [http://etherpad.mozilla.org:9000/ep/pad/view/ro.zIxyOfHFden/rev.259](http://etherpad.mozilla.org:9000/ep/pad/view/ro.zIxyOfHFden/rev.259) > It is really unfortunate that faceting can't get at the data the way we want. The only way I've thought of to really get it to work would be to have a flat array instead of a histogram. But that would be very expensive in terms of document size since: 5:5000 would turn into 5000 5 elements in the array. > > -Daniel > > On Wed, Jul 6, 2011 at 12:56 PM, Daniel Einspanjer \<[deinspanjer@gmail.com](mailto:deinspanjer@gmail.com) ([mailto:deinspanjer@gmail.com](mailto:deinspanjer@gmail.com))\> wrote: > > > I managed to make some progress.. I was able to create a script\_field that expands out all the data in each document's histogram: > > > > { > > "script\_fields": { > > "early\_gluestartup\_read\_ops": { > > "script": "var arr = ; var start = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[0]; var end = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.range[1]; var vals = \_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values; for (i = start; i \<= end; i++) { if (vals.containsKey(String.valueOf(i))) { for (j = 0; j \< vals[String.valueOf(i)]; j++) { arr.add(i); } } }; return arr;" > > } > > }, > > "query": { > > "term": { > > "OS": "winnt" > > } > > }, > > "facets": { > > "early\_gluestartup\_read\_ops": { > > "terms": { > > "field": "early\_gluestartup\_read\_ops" > > } > > } > > } > > } > > > > But I can't facet on the script field. I expect this is the same problem that was just posted in the mailing list where the person wanted to use a script\_field in a term\_stats. Think I've hit a dead end here. Gonna have to do these aggregations in some other system instead. > > > > -Daniel > > > > On Wed, Jul 6, 2011 at 11:06 AM, Daniel Einspanjer \<[deinspanjer@gmail.com](mailto:deinspanjer@gmail.com) ([mailto:deinspanjer@gmail.com](mailto:deinspanjer@gmail.com))\> wrote: > > > > > Ultimately, I was hoping to get a facet histogram of the histogram data stored in these documents. for instance, in each document, there is an EARLY\_GLUESTARTUP\_READ\_OPS.values object which has properties consisting of the histogram data, key is the histogram bucket and value is the count of entries for that key. > > > > > > It seems to be impossible to facet this data in any useful way, even with the use of script\_fields because I can't enumerate all the property keys under the .values object since I don't know them ahead of time due to their being a histogram. > > > > > > As an example, if I had the following three documents: > > > > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1}}} > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":2,"10":9,"11":0},"sum":1}}} > > > {"histograms":{"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"3":1,"5":11,"11":0},"sum":1}}} > > > > > > I would like to have some efficient way to generate the following aggregate result: > > > > > > {"sum\_histogram":{"0":0,"1":1,"3":3,"5":11,"10":9,"11":0}} > > > > > > But depending on the corpus of documents, I don't know what values will actually be enumerated. > > > > > > -Daniel > > > > > > On Tue, Jul 5, 2011 at 9:27 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote: > > > > > > > Maybe you can explain what you are trying to do exactly? I did not understand... > > > > > > > > On Wednesday, July 6, 2011 at 2:52 AM, Daniel Einspanjer wrote: > > > > > > > > > As a follow-up, I've been trying to use Elasticsearch Head's transformation function to aggregate the results, but the problem there is that I have to get the complete result set which exhausts memory in ES. > > > > > So, I've been trying to come up with a script\_fields statement that would transform the values object into a format that would allow me to facet on terms and then have a single object returned, but I can't find a way to do even that. here is the closest I've come which fails with an error that java.util.HashMap is not iterable: > > > > > > > > > > "script\_fields": { > > > > > "early\_gluestartup\_read\_ops": { > > > > > "script": "var arr = ; for (key : _source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values) { arr.push(key); }; arr.reduce(function(comb,key){return comb.push(key+'_'+\_source.histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values[key]);},); return comb;" > > > > > } > > > > > } > > > > > > > > > > On Tue, Jul 5, 2011 at 5:02 PM, Daniel Einspanjer \<[deinspanjer@gmail.com](mailto:deinspanjer@gmail.com) ([mailto:deinspanjer@gmail.com](mailto:deinspanjer@gmail.com))\> wrote: > > > > > > > > > > > Given documents with a structure like this: > > > > > > > > > > > > {"histograms":{"CYCLE\_COLLECTOR":{"range":[1,10000],"bucket\_count":50,"histogram\_type":0,"values":{"3":0,"4":4,"5":2,"7":1,"8":4,"10":2,"14":1,"17":0},"sum":102},"EARLY\_GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":1},"EARLY\_GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"27":0,"79":1,"232":0},"sum":154},"GLUESTARTUP\_READ\_OPS":{"range":[1,100],"bucket\_count":12,"histogram\_type":1,"values":{"0":0,"1":1,"11":0},"sum":2},"GLUESTARTUP\_READ\_TRANSFER":{"range":[1,51200],"bucket\_count":12,"histogram\_type":0,"values":{"0":1,"1":0},"sum":0},"MEMORY\_JS\_GC\_HEAP":{"range":[1024,524288],"bucket\_count":10,"histogram\_type":0,"values":{"4870":0,"10622":5,"23167":0},"sum":66560},"MEMORY\_LAYOUT\_ALL":{"range":[1024,65536],"bucket\_count":10,"histogram\_type":0,"values":{"0":0,"1024":4,"1722":1,"2896":0},"sum":8744},"MEMORY\_RESIDENT":{"range":[32768,1048576],"bucket\_count":10,"histog > > > > > > ram\_type":0,"values":{"50535":0,"77936":5,"120194":0},"sum":589172},"ZIPARCHIVE\_CRC":{"range":[1,2],"bucket\_count":3,"histogram\_type":2,"values":{"0":0,"1":87,"2":0},"sum":87}}} > > > > > > > > > > > > How could I use a script (such as in a facet query) to get at all the attributes under histograms.EARLY\_GLUESTARTUP\_READ\_OPS.values ? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/accessing-all-properties-of-an-object-field/4782/8 "2017-07-06T04:01:10Z")