# How to access IIS request source IP from within pipeline.conf? **URL:** \ **Category:** Logstash\ **Created:** [December 30, 2019, 11:02pm UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387 "2019-12-30T23:02:00Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![dRm42](https://avatars.discourse-cdn.com/v4/letter/d/b4bc9f/32.png) [@dRm42](https://discuss.elastic.co/u/dRm42)\ **Post date:** [December 30, 2019, 11:02pm UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387/1 "2019-12-30T23:02:00Z")
Hi Guys, I've spent more than a day trying to figure this out but just couldn't get it working. I'm using Filebeat to collect IIS logs, push them through Logstash and then into ES. Filebeat has the IIS module enabled and everything seems to be working until I attempt to enrich the entries with Geoip information. Lots of our requests are coming from the internal network. Geoip fails on every private IP address (obviously), but I need to map out internal requests as well, so I have to make the geoip plugin params conditional. Instead of forging the geoip info field-by-field, I've decided to simply change the source address to our external IP, that geoip can resolve. This worked like a charm but, unexpectedly, I ran into a concrete wall with the condition itself. logstash.yml > xpack.monitoring.enabled: true > xpack.monitoring.elasticsearch.hosts: "[http://SOMESERVER:9200](http://SOMESERVER:9200)" > > pipeline.id: beats > path.config: "../config/beats.conf" > pipeline.workers: 4 > pipeline.batch.size: 500 > > config.reload.automatic: true > queue.type: persisted > queue.max\_bytes: 8gb beats.conf > input { > beats { > port =\> 5044 > } > } > > filter { > if "iis" in [tags] { > if [c-ip] =~ /^10./ or [c-ip] =~ /^192./ { > mutate { add\_field =\> { "external-ip" =\> "xxx.xxx.xxx.xxx" } } > geoip { > source =\> "external-ip" > target =\> "geoip" > add\_tag =\> ["internal\_request"] > } > mutate { remove\_field =\> ["external-ip"] } > } > else { > geoip { > source =\> "c-ip" > target =\> "geoip" > add\_tag =\> ["external\_request"] > } > } > } > } > > output { > if "iis" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "iis-%{+YYYY.MM.dd}" > pipeline =\> "%{[@metadata][pipeline]}" > } > } > else if "event\_log" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "event\_log-%{+YYYY.MM.dd}" > } > } > else if "elasticsearch\_log" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "elasticsearch\_log-%{+YYYY.MM.dd}" > } > } > else if [@metadata][pipeline] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "unidentified-%{+YYYY.MM.dd}" > } > } > else { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "other-%{+YYYY.MM.dd}" > } > } > } The problem is that the IF statement always falls into the ELSE path, not matter what I do. I've tried simple regex: > if [c-ip] =~ /^10./ or [c-ip] =~ /^192./ { > ... I've tried complex regex: > if [c-ip] =~ /^((127.)|(10.)|(172.1[6-9].)|(172.2[0-9].)|(172.3[0-1].)|(192.168.))/ { > ... > ...these all land in the ELSE path. I've tried using CIDR as someone recommended here on the forum: > ``` > cidr { > address => ["%{c-ip}"] > network => ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"] > add_field => { "source.network" => "internal" } > } > ... > > ``` ...or: > cidr { > address =\> ["[c-ip]" ] > network =\> ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"] > add\_field =\> { "source.network" =\> "internal" } > } > ... ...or: > cidr { > address =\> ["c-ip"] > network =\> ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"] > add\_field =\> { "source.network" =\> "internal" } > } > ... They all failed with: > [WARN][logstash.filters.cidr][beats] Invalid IP address, skipping... This was the point where started thinking that maybe it's not my regex or the CIDR call that is faulty, but "c-ip" simply isn't available as a field so I'm comparing NULL every time. However, this works just fine (although it's not conditional as I'd like): > ... > filter { > geoip { > source =\> "c-ip" > target =\> "geoip" > add\_tag =\> ["geoip"] > } > } > ... ...so "c-ip" is in fact available in some form or shape, just not directly from the filter plugin. I've also tried [source][address] and [source][ip] none of them seemed to have worked. Without these, I can't filter in my conditional and can't utilize geoip properly. So my question is, how can I access the source IP of an IIS request from within the filter section of a config file. If someone could answer this, it would really make my New Year 🙂 Cheers, Laz ---
**Author:** ![dRm42](https://avatars.discourse-cdn.com/v4/letter/d/b4bc9f/32.png) [@dRm42](https://discuss.elastic.co/u/dRm42)\ **Post date:** [December 31, 2019, 1:33am UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387/2 "2019-12-31T01:33:44Z")
If someone happens to know the answer to my question, that is, why "c-ip" is accessible for the geoip plugin, but not for the filter plugin, or how is it possible to access the source IP from the filter plugin otherwise, I'd be still curious to know. Otherwise we can close this one. I think I've managed to put something together that works. Just GROK'd the source IP out of the message field and used that in the address field of the cidr plugin. In case someone runs into the same problem in the future: > input { > beats { > port =\> 5044 > } > } > > filter { > if "iis" in [tags] { > grok { > match =\> { "message" =\> "%{TIMESTAMP\_ISO8601} %{IPV4} %{WORD} %{URIPATH} %{NOTSPACE} %{INT} %{NOTSPACE} %{IPV4:originator} %{GREEDYDATA}"} > } > > cidr { > address =\> ["%{originator}"] > network =\> ["10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8"] > add\_field =\> { "source.network" =\> "internal" } > } > > ``` > if [source.network] == "internal" { > mutate { add_field => { "external-ip" => "xxx.xxx.xxx.xxx" } } > geoip { > source => "external-ip" > target => "geoip" > tag_on_failure => ["geoip_incomplete"] > } > mutate { remove_field => ["external-ip"] } > } > else { > geoip { > source => "c-ip" > target => "geoip" > tag_on_failure => ["geoip_incomplete"] > add_field => { "source.network" => "external" } > } > } > > ``` > > } > mutate { > remove\_tag =\> ["beats\_input\_codec\_plain\_applied"] > remove\_field =\> ["originator", "agent", "[geoip][ip]", "[ecs][version]", "[host][id]", "[host][hostname]", "[host][os][family]", "[host][os][platform]", "[host][os][version]" ] > } > } > > output { > if "iis" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "iis-%{+YYYY.MM.dd}" > pipeline =\> "%{[@metadata][pipeline]}" > } > } > else if "event\_log" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "event\_log-%{+YYYY.MM.dd}" > } > } > else if "elasticsearch\_log" in [tags] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "elasticsearch\_log-%{+YYYY.MM.dd}" > } > } > else if [@metadata][pipeline] { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "unidentified-%{+YYYY.MM.dd}" > } > } > else { > elasticsearch { > hosts =\> ["[http://SOMESERVER:9200](http://SOMESERVER:9200)"] > index =\> "other-%{+YYYY.MM.dd}" > } > } > } ---
**Author:** ![dRm42](https://avatars.discourse-cdn.com/v4/letter/d/b4bc9f/32.png) [@dRm42](https://discuss.elastic.co/u/dRm42)\ **Post date:** [December 31, 2019, 1:35am UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387/3 "2019-12-31T01:35:09Z")
Not sure why my posts look like this in blockquote, sorry about that. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 28, 2020, 1:35am UTC](https://discuss.elastic.co/t/how-to-access-iis-request-source-ip-from-within-pipeline-conf/213387/4 "2020-01-28T01:35:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.