# CIDR, location lookup in the most efficient way

**URL:** <https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232>\
**Category:** Logstash\
**Created:** [May 9, 2018, 10:01pm UTC](https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232 "2018-05-09T22:01:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![geertn444](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geertn444/32/31289_2.png) [@geertn444](https://discuss.elastic.co/u/geertn444)\
**Post date:** [May 9, 2018, 10:01pm UTC](https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232/1 "2018-05-09T22:01:38Z")

</div>

I want to map IPs in a certain range to a site. I don't want to map the IP ranges inside logstash config files, but in external files. CIDR has this possibility using network\_path

So now i have a logic like this:

if [flow][dst\_addr] {  
cidr {  
address =\> ["%{[flow][dst\_addr]}" ]  
network\_path =\> "/local/etc/subnets/a.txt"  
add\_field =\> { "[flow][dst\_site]" =\> "A" }  
}

cidr {  
address =\> ["%{[flow][dst\_addr]}" ]  
network\_path =\> "/local/etc/subnets/B.txt"  
add\_field =\> { "[flow][dst\_site]" =\> "B" }  
}  
cidr {  
address =\> ["%{[flow][dst\_addr]}" ]  
network\_path =\> "/local/etc/subnets/C.txt"  
add\_field =\> { "[flow][dst\_site]" =\> "C" }  
}  
cidr {  
address =\> ["%{[flow][dst\_addr]}" ]  
network\_path =\> "/local/etc/subnets/D.txt"  
add\_field =\> { "[flow][dst\_site]" =\> "D" }  
}

The first disadvantage of this is that logstash will loop through ALL cidr matches, even when a match is found in the first CIDR filter. We should create a field or variable, called "found" which is false in the beginning, but once a match is found, it is set to true and all subsequent cidr matches are skipped. Is this possible with tags ?

The second disadvantage of this is that the smallest match isn't necesarrily returned.  
For example, consider following example:

A.txt = 10.1.0.0/16  
B.txt = 10.1.1.0/24

An ip 10.1.1.1 will match A and not do B anymore, while B is a more accurate match.  
This is caused by the fact that our search is split across multiple files.  
The best would be to create and look into a single file for all subnets.  
Subnet searches could then be optimised from small to large for example.

10.1.1.0/24 : site B  
10.1.0.0/16 : site A

the lookup can be ordered or optimised in this case by ordering subnets internally from /32 match (small) to largest blocks (/19, etc).

We could read multiple files at once, with something like:

cidr {  
address =\> ["%{[flow][dst\_addr]}" ]  
network\_path =\> "/local/etc/subnets/A.txt"  
network\_path =\> "/local/etc/subnets/B.txt"  
network\_path =\> "/local/etc/subnets/C.txt"  
#ordered from small to large in memory to prevent above problem  
add\_field =\> { "[flow][dst\_site]" =\> "{%filename}" }  
}

regards,  
Geert

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 16, 2018, 8:18pm UTC](https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232/3 "2018-05-16T20:18:28Z")

</div>

> The first disadvantage of this is that logstash will loop through ALL cidr matches, even when a match is found in the first CIDR filter. We should create a field or variable, called "found" which is false in the beginning, but once a match is found, it is set to true and all subsequent cidr matches are skipped. Is this possible with tags ?

Yes. Add `add_tag => "found"` to you cidr filters and wrap each filter in `if "found" not in [tags] { ... }`.

> We could read multiple files at once, with something like:

Why not just generate a single file that you feed a single cidr filter?

---

<div class="post-metadata">

**Author:** ![geertn444](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/geertn444/32/31289_2.png) [@geertn444](https://discuss.elastic.co/u/geertn444)\
**Post date:** [May 16, 2018, 8:59pm UTC](https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232/4 "2018-05-16T20:59:57Z")

</div>

> Why not just generate a single file that you feed a single cidr filter?

Because i want to set a field differently based on which subnet is matched.  
ie match on subnet A, B or C -\> set field X  
match on subnet D -\> set field Y

or as in my example:  
add\_field =\> { "[flow][dst\_site]" =\> "{%filename}" } \<-- filename determines string that is set

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2018, 9:00pm UTC](https://discuss.elastic.co/t/cidr-location-lookup-in-the-most-efficient-way/131232/5 "2018-06-13T21:00:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.