Deep Aggregation Support


(Robert George) #1

Hi.

I do have a question.

Does elasticsearch support deep aggregations? (nested aggregation in nested aggregation). To make agg of a nested object inside a nested object. Specifically to count docs.
If so, please give me an example.

Thank you very much.


(Mark Harwood) #2

"Nested" can be a reserved word in elasticsearch. Do you mean in the formal sense of nested field types and the nested type of aggregation or the more general sense of nesting JSON expressions?


(Robert George) #3

Let me give an example:
Suppose you have a ES document like this:
{
IP:[{
Address: ip,
geoip: {
Country: string
}
},
{...}]
}
Both geoip and IP are nested objects

I want to count the ips (from the array field of nested objects IP), based on their country.
USA: 5
Great Britain:2

Is this doable?
Thank you.


(Mark Harwood) #4

Example mapping docs and query:

DELETE test
PUT test
{
  "mappings":{
	"doc":{
	  "properties": {
		"IP":{
		  "type":"nested",
		  "properties": {
			"address":{
			  "type":"ip"
			},
			"country":{
			  "type":"keyword"
			}
		  }
		}
	  }
	}
  }
}
POST test/doc/_bulk
{"index":{}}
{"IP":[ {"address":["1.1.1.1","1.1.1.2"], "country":"c1"}]}
{"index":{}}
{"IP":[ {"address":["2.2.2.2"], "country":"c2"}]}

POST test/_search
{
  "size":0,
  "aggs": {
	"byCountry": {
	  "nested": {
		"path": "IP"
	  },
	  "aggs": {
		"country": {
		  "terms": {
			"field": "IP.country"
		  },
		  "aggs":{
			"ipCount":{
			  "cardinality":{
				"field":"IP.address"
			  }
			}
		  }
		}
	  }
	}
  }
}

(Robert George) #5

Thank you for this reply but the IP nested object that has an geoip nested object that contains country keywork, like this:

 DELETE test
        PUT test
        {
          "mappings":{
        	"doc":{
        	  "properties": {
        		"IP":{
        		  "type":"nested",
        		  "properties": {
        			"address":{
        			       "type":"ip"
        			},
        			"geoip":{
        			   "type":"nested",
        		           "properties": {
        			       "country":{
        			              "type":"keyword"
        			          }
        		            }
        			}
        		  }
        		}
        	  }
        	}
          }
        }

(Robert George) #6

I have this mapping because of geoip plugin, which makes a geoip nested object.

Or is there a way to make geoip plugin not to make a geoip nested object and instead to include the fields like you did there?


(Mark Harwood) #7

I'm not familiar with the format of the geoip plugin but you can use an ingest pipeline to manipulate JSON prior to indexing where required.


(Robert George) #8

Ok. Thank you


(Robert George) #9

Could you give me an example of using 2 pipelines in a row? Like these 2: geoip and ingest
I am writting querries with kibana directly for ES.

Thank you for your help


(Mark Harwood) #10

My understanding is a single pipeline can include multiple processors and the geoip plugin describes itself as a processor so should be configurable as one of a sequence of processors in a pipeline.


(Robert George) #11

Thank you. That's all.


(system) #12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.