Translate private ip to geo location using external dictionary

[ptamba]

Hi all,

I'm trying to follow example in this link https://discuss.elastic.co/t/private-ip-geoip-from-dictionary/156763, however I can't seem to make it work. Basically I want to translate private ip address using external dictionary path.

Here's my translate filter config :

 translate {
      exact => true
      regex => true 
      field => "[source][ip]"
      destination => "[source][geo]"
      dictionary_path => "./geo.yml"
    }

And here's the dictionary file :

`'10.5.181.74': '{"geoip":{"timezone":"Asia/Jakarta","continent_code":"NA","country_name":"Indonesia","region_code":"JK","country_code2":"ID","country_code3":"ID","region_name":"Jakarta","city_name":"Jakarta","latitude":-6.196459,"longitude":106.822451,"location":{"lat":-6.196459,"lon":106.822451}}}'`

I'm expecting the geo to be mapped to source.geo field, but it didn't. this is the output.

    "source" => {
            "geo" => "{\"geoip\":{\"timezone\":\"Asia/Jakarta\",\"continent_code\":\"NA\",\"country_name\":\"Indonesia\",\"region_code\":\"JK\",\"country_code2\":\"ID\",\"country_code3\":\"ID\",\"region_name\":\"Jakarta\",\"city_name\":\"Jakarta\",\"latitude\":-6.196459,\"longitude\":106.822451,\"location\":{\"lat\":-6.196459,\"lon\":106.822451}}}",
             "ip" => "10.5.181.74"
        },

any pointers?

[Badger]

That looks like it worked perfectly. What is the problem?

[ppafford (Phill Pafford)]

I think it's your translate, it looks to be doing what you coded but I think you're wanting this to be a geoip location. se the translation Im doing here, I think this will put you on the right path Private ip geoip from dictionary

destination => "geo_point"

[ptamba]

the translation worked, but it stored the geo coordinates as string rather than a geo location

[ptamba]

nope, still didn't solve it. trying your suggestion creates a new field called "geo_point" with the value of coordinates as a string :

if "src_internal_ip" in [tags] {
    translate {
      exact => true
      regex => true 
      field => "[source][ip]"
      destination => "geo_point"
      dictionary_path => "./geo.yml"
    }
  }

output :

"geo_point" => "{\"geoip\":{\"timezone\":\"Asia/Jakarta\",\"continent_code\":\"NA\",\"country_name\":\"Indonesia\",\"region_code\":\"JK\",\"country_code2\":\"ID\",\"country_code3\":\"ID\",\"region_name\":\"Jakarta\",\"city_name\":\"Jakarta\",\"latitude\":-6.196459,\"longitude\":106.822451,\"location\":{\"lat\":-6.196459,\"lon\":106.822451}}}",
       "fw_rule_id" => "0",

[ptamba]

my mistake, i need to parse the resulting json using json filter. missed this part from the original post. here's working config :

if "src_internal_ip" in [tags] {
    translate {
      exact => true
      regex => true 
      override => true
      refresh_behaviour => "replace"
      field => "[source][ip]"
      destination => "geo_point"
      dictionary_path => "./geo.yml"
    }

    json {
      source => "geo_point"
      target => "[source][geo]"
    }
  }

output:

"source" => {
         "ip" => "10.5.181.74",
        "geo" => {
            "geoip" => {
                     "city_name" => "Jakarta",
                   "region_code" => "JK",
                     "longitude" => 106.822451,
                      "location" => {
                    "lat" => -6.196459,
                    "lon" => 106.822451
                },
                  "country_name" => "Indonesia",
                 "country_code3" => "ID",
                      "latitude" => -6.196459,
                   "region_name" => "Jakarta",
                 "country_code2" => "ID",
                      "timezone" => "Asia/Jakarta",
                "continent_code" => "NA"
            }
        }

thanks @ppafford for the post