# Using bucket aggregation on IPs not giving expected results

**URL:** <https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165>\
**Category:** Elasticsearch\
**Created:** [September 12, 2017, 7:30am UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165 "2017-09-12T07:30:29Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 12, 2017, 7:30am UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/1 "2017-09-12T07:30:29Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/f/2/f2cfd2d5b94848b3f46aaf6065e94f4bed134930.png)

Hello,

I am trying to search for documents containing an IP which falls under a range. As specified in the image the docs have IP 192.168.255.100 which falls under the mask: 192.168.0.0/24. But the result that I get is not as per my understanding. I was expecting to get docs in the bucket. Moreover if I try with 192.168.0.0/24 I get ArrayOutOfBoundIndexException. Not sure what's happening. Also from and to string is weird with some different  
encoding.

Could you please point me out as to where its getting wrong?

Thanks

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 13, 2017, 5:20pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/2 "2017-09-13T17:20:51Z")

</div>

The name of the field that you run this aggregation against (`ipAddress.keyword`) suggests that you are running this aggregation against a field that has been mapped as type `keyword`. You need to map this field as[type `ip`](https://www.elastic.co/guide/en/elasticsearch/reference/current/ip.html) if you want to use the `ip_range` aggregation.

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 7:09am UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/3 "2017-09-14T07:09:57Z")

</div>

If I use ip type in aggregation it gives me error saying **"Fielddata is disabled on text fields by default. Set fielddata=true on [ipAddress] in order to load fielddata in memory by uninverting the inverted index"**

I don't want to use fielddata because it would be costly in terms of memory. So I opted for multi-field and used keyword subfield to use in aggs.

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 14, 2017, 8:37am UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/4 "2017-09-14T08:37:58Z")

</div>

I think your field `ipAddress` is actually mapped as a type `text` rather than as type `ip`. You could check that by retrieving the mapping:

`GET hostaddressindex/client/_mapping`

I think you will see `"type": "text"` instead of `"type": "ip"` for the `ipAddress` field.

You will have to change your mapping such that `ipAddress` is mapped as type `ip`. Note that you cannot change the mapping (including the type) of an existing field. What you need to do if you want to change the mapping of existing documents is reindex those documents to another index with the updated mapping.

So, first create a new index, which you create with the new mapping. Then use the reindex API to get all documents from the old index A into the new index B. As those documents get reindexed, they will get the updated mapping applied to them. More info about the reindex API here: [https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 11:09am UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/5 "2017-09-14T11:09:44Z")

</div>

I remember I had the type set to ip. Let me clean everything up and start over again and re-confirm it.

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 12:27pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/6 "2017-09-14T12:27:14Z")

</div>

Hello Abdon,

Here is what I tried in the mentioned sequence.  
Query: GET /hostaddressindex/host/\_mapping  
Response:  
{  
"hostaddressindex": {  
"mappings": {  
"host": {  
"\_all": {  
"enabled": false  
},  
"properties": {  
"ipAddress": {  
"type": "ip",  
"fields": {  
"raw": {  
"type": "text"  
}  
}  
},  
"link": {  
"type": "text",  
"index": false  
},  
"timeInterval": {  
"type": "integer",  
"index": false  
}  
}  
}  
}  
}  
}

Query: POST /hostaddressindex/host/1  
{  
"ipAddress" : "192.168.1.1",  
"link" : "192.168.1.1|3",  
"timeInterval" : [1]  
}

Response:  
{  
"\_index": "hostaddressindex",  
"\_type": "host",  
"\_id": "1",  
"\_version": 4,  
"result": "updated",  
"\_shards": {  
"total": 1,  
"successful": 1,  
"failed": 0  
},  
"created": false  
}

Query:GET /hostaddressindex/host/1  
Response:  
{  
"\_index": "hostaddressindex",  
"\_type": "host",  
"\_id": "1",  
"\_version": 4,  
"found": true,  
"\_source": {  
"ipAddress": "192.168.1.1",  
"link": "192.168.1.1|3",  
"timeInterval": [  
1  
]  
}  
}

Query:  
GET \_search  
{  
"aggs" : {  
"ip\_ranges" : {  
"ip\_range" : {  
"field" : "ipAddress.raw",  
"ranges" : [  
{  
"mask" : "192.168.0.0/16"  
}  
]  
}  
}  
}  
}

Response:  
{  
"took": 1,  
"timed\_out": false,  
"\_shards": {  
"total": 10,  
"successful": 5,  
"failed": 5,  
"failures": [  
{  
"shard": 0,  
"index": "indexing",  
"node": "qzkSbw6dTDugoLnF5mECKw",  
"reason": {  
"type": "illegal\_argument\_exception",  
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [ipAddress] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."  
}  
}  
]  
},  
"hits": {  
"total": 1,  
"max\_score": 1,  
"hits": [  
{  
"\_index": "hostaddressindex",  
"\_type": "host",  
"\_id": "1",  
"\_score": 1,  
"\_source": {  
"ipAddress": "192.168.1.1",  
"link": "192.168.1.1|3",  
"timeInterval": [  
1  
]  
}  
}  
]  
},  
"aggregations": {  
"ip\_ranges": {  
"buckets": [  
{  
"key": "192.168.0.0/16",  
"from": "192.168.0.0",  
"to": "192.169.0.0",  
"doc\_count": 1  
}  
]  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 14, 2017, 12:56pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/7 "2017-09-14T12:56:44Z")

</div>

Instead of running the aggregation against `ipAddress.raw` (which is mapped as type `text`), you need to run it against `ipAddress` (which is mapped as type `ip`). The following should work now:

```
GET _search
{
  "size": 0,
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "ipAddress",
        "ranges": [
          {
            "mask": "192.168.0.0/16"
          }
        ]
      }
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 1:11pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/8 "2017-09-14T13:11:30Z")

</div>

I am sorry I pasted the wrong thing. It is ipAddress instead of ipAddress.raw. Still I get the issue.This is the query I fired:  
GET \_search  
{  
"aggs" : {  
"ip\_ranges" : {  
"ip\_range" : {  
"field" : "ipAddress",  
"ranges" : [  
{  
"mask" : "192.168.0.0/16"  
}  
]  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 14, 2017, 1:14pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/9 "2017-09-14T13:14:17Z")

</div>

Can you please post the output of:

`GET /hostaddressindex/_mapping`

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 2:03pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/10 "2017-09-14T14:03:50Z")

</div>

{  
"hostaddressindex": {  
"mappings": {  
"server": {  
"\_all": {  
"enabled": false  
},  
"properties": {  
"ipAddress": {  
"type": "ip",  
"fields": {  
"raw": {  
"type": "text"  
}  
}  
},  
"link": {  
"type": "text",  
"index": false  
},  
"timeInterval": {  
"type": "integer",  
"index": false  
}  
}  
},  
"client": {  
"\_all": {  
"enabled": false  
},  
"properties": {  
"ipAddress": {  
"type": "ip",  
"fields": {  
"raw": {  
"type": "text"  
}  
}  
},  
"link": {  
"type": "text",  
"index": false  
},  
"timeInterval": {  
"type": "integer",  
"index": false  
}  
}  
},  
"host": {  
"\_all": {  
"enabled": false  
},  
"properties": {  
"ipAddress": {  
"type": "ip",  
"fields": {  
"raw": {  
"type": "text"  
}  
}  
},  
"link": {  
"type": "text",  
"index": false  
},  
"query": {  
"properties": {  
"match\_all": {  
"type": "object"  
}  
}  
},  
"timeInterval": {  
"type": "integer",  
"index": false  
}  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\
**Post date:** [September 14, 2017, 2:16pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/11 "2017-09-14T14:16:46Z")

</div>

The error is coming from an index named `indexing` which you are searching because you are doing `GET _search` rather than `GET /hostaddressindex/_search`. The index `indexing` probably also has a field call `ipAddress` but that one is mapping to `text`. You need to search just the index you want the data from, or if you want data from both indexes you will need to re-index `indexing` with the `ipAddress` field mapping as the `ip` type

---

<div class="post-metadata">

**Author:** ![Sambit\_Kabi](https://avatars.discourse-cdn.com/v4/letter/s/bbce88/32.png) [@Sambit\_Kabi](https://discuss.elastic.co/u/Sambit_Kabi)\
**Post date:** [September 14, 2017, 2:26pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/12 "2017-09-14T14:26:15Z")

</div>

My bad...I didn't realise this and just skipped the indexing index in the error.

Thanks Colin and Abdon

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 12, 2017, 2:26pm UTC](https://discuss.elastic.co/t/using-bucket-aggregation-on-ips-not-giving-expected-results/100165/13 "2017-10-12T14:26:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.