Geoip is not generate .location

Elastic Stack Logstash
21

Hi Christian

Changes I did:

pam-traffic.json

"properties" : {
         "@version": { "type": "string", "index": "not_analyzed" },
         "geoip"  : {
           "type" : "object",
             "dynamic": true,
             "properties" : {
                "ip": { "type": "ip" }, "location" : { "type" : "geo_point" }, "latitude" : { "type" : "half_float" }, "longitude" : { "type" : "half_float" }
             }
         },
        "SourceGeo"  : {
           "type" : "object",
             "dynamic": true,
             "properties" : {
               "location" : { "type" : "geo_point" }
             }
         },
         "DestinationGeo"  : {
           "type" : "object",
             "dynamic": true,
             "properties" : {
               "location" : { "type" : "geo_point" }
             }
         }

curl -XPUT 'http://localhost:9200/_template/pan-traffic' -d@/etc/logstash/pantraffic-template.json
curl -XDELETE 'localhost:9200/pan-traffic?pretty'

Delete the pan-index in kibana and regenerate it and the result is

Captura.JPG1042×178 18.8 KB

22

any ideas why does not work properly?

kindly regards

23

Good morning,

I've reviewed the output of the logstash and location has lon and lat as array as you can see in the following trace:

"DestinationGeo":{  
  "timezone":"America/Los_Angeles",
  "ip":"216.58.206.142",
  "latitude":37.419200000000004,
  "continent_code":"NA",
  "city_name":"Mountain View",
  "country_name":"United States",
  "country_code2":"US",
  "dma_code":807,
  "country_code3":"US",
  "region_name":"California",
  "location":{  
     "lon":-122.0574,
     "lat":37.419200000000004
  },
  "postal_code":"94043",
  "region_code":"CA",
  "longitude":-122.0574

But in kibana doesnt appear the option related to DestionationGeo.location :S

the template loaded into elasticsearch is the following:

{
  "pan-traffic" : {
    "order" : 0,
    "version" : 2,
    "template" : "pan_traffic",
    "settings" : {
      "index" : {
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "_default_" : {
        "dynamic_templates" : [
          {
            "message_field" : {
              "mapping" : {
                "index" : "analyzed",
                "omit_norms" : true,
                "type" : "string"
              },
              "match_mapping_type" : "string",
              "match" : "message"
            }
          },
          {
            "string_fields" : {
              "mapping" : {
                "index" : "analyzed",
                "omit_norms" : true,
                "type" : "string",
                "fields" : {
                  "raw" : {
                    "ignore_above" : 256,
                    "index" : "not_analyzed",
                    "type" : "string"
                  }
                }
              },
              "match_mapping_type" : "string",
              "match" : "*"
            }
          }
        ],
        "_all" : {
          "enabled" : true
        },
        "properties" : {
          "SourceGeo" : {
            "dynamic" : true,
            "properties" : {
              "latitude" : {
                "type" : "half_float"
              },
              "location" : {
                "type" : "geo_point"
              },
              "longitude" : {
                "type" : "half_float"
              }
            }
          },
          "DestinationGeo" : {
            "dynamic" : true,
            "properties" : {
              "latitude" : {
                "type" : "half_float"
              },
              "location" : {
                "type" : "geo_point"
              },
              "longitude" : {
                "type" : "half_float"
              }
            }
          },
          "@version" : {
            "index" : "not_analyzed",
            "type" : "string"
          }
        }
      }
    },
    "aliases" : { }
  }
}

the logstash.conf is the following

if [SourceAddress] and [SourceAddress] !~ "(^127\.0\.0\.1)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)|(^169\.254\.)" {
      geoip {
           database => "/etc/logstash/GeoLite2.mmdb"
           source => "SourceAddress"
           target => "SourceGeo"
      }
      #Delete 0,0 in SourceGeo.location if equal to 0,0
      if ([SourceGeo.location] and [SourceGeo.location] =~ "0,0") {
        mutate {
          replace => [ "SourceGeo.location", "" ]
        }
      }
    }
   #Geolocate logs that have DestinationAddress and if that DestinationAddress is a non-RFC1918 address
  if [DestinationAddress] and [DestinationAddress] !~ "(^127\.0\.0\.1)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)|(^169\.254\.)" {
      geoip {
           database => "/etc/logstash/GeoLite2.mmdb"
           source => "DestinationAddress"
           target => "DestinationGeo"
      }
      #Delete 0,0 in DestinationGeo.location if equal to 0,0
      if ([DestinationGeo.location] and [DestinationGeo.location] =~ "0,0") {
        mutate {
          replace => [ "DestinationAddress.location", "" ]
        }
      }
    }

Any ideas why it's not generating the .location as geo_point as DestinationGeo and SourceGeo

24

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.