TileMap, GeoIP for RFC1918 addresses

Elastic Stack Logstash
1

Hi Community,

looking for an opportunity to locate rfc1918 addresses to a tile map.

Non-rfc1918 works fine via GeoIP Database:

geoip {
         source =>   "client_address"
         target =>   "geoip"
         database => "/etc/logstash/conf.d/geo/GeoLiteCity.dat"
         add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
         add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
         }

      mutate {
         convert => [ "[geoip][coordinates]", "float"]
         }

Is it possible to catch specific rfc1918 addresses and manipulate with specififc latitude, lontidtude -> geo_hash values.

Or add rfc1918 addresses to GeoLiteCity.dat ?

Thank you

2

You may want to explain what that is.

Have a look at Creating geoip data for internal networks

1 Like
3

geoip {

     source =>   "client_address"
     target =>   "geoip"
     database => "/etc/logstash/conf.d/geo/GeoLiteCity.dat"
     add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
     add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]

        if [client_address] =~ /^10\./ {
           mutate { replace      => { "[geoip][timezone]"      => "Pacific/Auckland" } }
           mutate { replace      => { "[geoip][country_name]"  => "University of Otago" } }
           mutate { replace      => { "[geoip][country_code2]" => "UO" } }
           mutate { replace      => { "[geoip][country_code3]" => "UoO" } }
           mutate { remove_field => [ "[geoip][location]" ] }
           mutate { add_field    => { "[geoip][location]"      => "170.525" } }
           mutate { add_field    => { "[geoip][location]"      => "-45.865" } }
           mutate { convert      => [ "[geoip][location]",        "float" ] }
           mutate { replace      => [ "[geoip][latitude]",        -45.856 ] }
           mutate { convert      => [ "[geoip][latitude]",        "float" ] }
           mutate { replace      => [ "[geoip][longitude]",       170.525 ] }
           mutate { convert      => [ "[geoip][longitude]",       "float" ] }
        }

     }

Dosen't work:

The given configuration is invalid. Reason: Expected one of #, => at line 34, column 8 (byte 2419) after filter {
        if [type] == "syslog" {
                syslog_pri { }
                date {
                        match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
                }
                mutate {
                replace => [ "host", "%{logsource}" ]

        }
                if [program] == "accesslog-elastic" {
                        grok {
            patterns_dir => ["/etc/logstash/conf.d/patterns"]
                                match => [
                                        "message" ,"%{WORD:message_type}: %{NUMBER:log_time} %{NUMBER:duration} %{IP:client_address} %{WORD:transaction_result_code}/%{NUMBER:http_result_code} %{NUMBER:bytes:int} %{WORD:http_metod} %{PROTOCOL:url_protocol}%{DOMAIN:url_domain}%{REFERER:url_referer} %{NOTSPACE:user} %{NOTSPACE:requested_server} %{NOTSPACE:response_mime_type} %{NOT_HYPHEN:acl_decision_tag}-%{NOT_HYPHEN:access_or_decryption_policy}-%{NOT_HYPHEN:identity_policy_group}-%{NOT_HYPHEN:outbound_maleware_scanning_policy_group}-%{NOT_HYPHEN:data_security_policy_group}-%{NOT_HYPHEN:external_dlp_policy_group}-%{NOT_HYPHEN:routing_policy_group} <%{NOT_COMMA:url_category},%{NOT_COMMA:wbrs},%{NOT_COMMA:webroot_verdict},%{NOT_COMMA:spyname},%{NOT_COMMA:trr},%{NOT_COMMA:threat_id},%{NOT_COMMA:trace_id},%{NOT_COMMA:mcafee_verdict},%{NOT_COMMA:mcafee_filenmae},%{NOT_COMMA:mcafee_scan_error_code},%{NOT_COMMA:mcafee_detection_type},%{NOT_COMMA:mcafee_virus_type},%{NOT_COMMA:mcafee_virus_name},%{NOT_COMMA:sophos_verdict},%{NOT_COMMA:sophos_scan_return_code},%{NOT_COMMA:sophos_file_location},%{NOT_COMMA:sophos_threat_name},%{NOT_COMMA:data_security},%{NOT_COMMA:data_loss_prevention},%{NOT_COMMA:requested_side_url_verdict},%{NOT_COMMA:response_side_url_verdict},%{NOT_COMMA:unified_inbound_dvs_verdict},%{NOT_COMMA:web_reputation_filter_type},%{NOT_COMMA:avc_application_name},%{NOT_COMMA:avc_application_type},%{NOT_COMMA:avc_application_behavior},%{NOT_COMMA:avc_safe_browsing_scanning_verdict},%{NOT_COMMA:average_bandwidth},%{NOT_COMMA:throttle_flag},%{NOT_COMMA:type_of_user},%{NOT_COMMA:unified_outbound_dvs_verdict},%{NOT_COMMA:outbound_threat_name}%{GREEDYDATA:message_body}>"
                                ]
                        }

                mutate {
                #convert => [ "bytes", "integer" ]
                        #convert => [ "duration", "integer" ]
                        #convert => [ "client_address", "ip" ]
                        }

                geoip {

                        source =>       "client_address"
                        target =>       "geoip"
                        database => "/etc/logstash/conf.d/geo/GeoLiteCity.dat"
                        add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
        add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]

                                if  {:level=>:fatal}
4

Please format your code with the </> button, it’s very hard to read.

5

Ok. Done.

6

RFC1918 is the Request for comment of private IPv4 addresses...

7

You can not have conditionals with a filter, which is why it is complaining about the configuration being invalid.

8

And how does : Creating geoip data for internal networks works ?

9

You need to put the filters within the conditionals. If you identify the IP starts with an internal prefix, run your mutate filters and if this is not the case run the geoip filter.

10

Please show me the an examlpe of this config...

11

Something like this:

if [client_address] =~ /^10\./ {
  mutate {}
} else {
  geoip {}
}

There is more on conditionals here.

1 Like
12

Works fine! Thank you!:slight_smile: