Manually Setting Geoip.Location

Elastic Stack Logstash
1

Hello Folks,

I am looking for a way to manually set the geo location of various Logstash servers. While these Logstash server are all separated geographically, they are collecting logs and beats from servers that don't have public IP connections. Individual user IPs aren't interesting for my use cases either (none are public), so the usual recommendations using the GeoLite2 City database won't work for me.

What I am attempting is to enrich the indexes with geoip.location being statically set. From ingest at Logstash, this is what I've done at the end of all the log filtering, using mutate to manually add the field. (my first attempt is commenetd out).

mutate {
  add_field => [ "[geoip][location]", "-79.3849" ]
  add_field => [ "[geoip][location]", "43.6529" ]
}

Here is the mapping I'm using in the template, which matches all of these log indexes.

"geoip": {
            "dynamic": true,
            "properties": {
              "ip": {
                "type": "ip"
              },
              "location": {
                "type": "geo_point"
              },
              "latitude": {
                "type": "half_float"
              },
              "longitude": {
                "type": "half_float"
              }

However, I can see that the geoip.location type appears as a string when I explore the index patterns for these logs.

geoip.location -> string

And when I Discover the data in the index pattern, I can see it present, but as a string type (should be a globe symbol I believe in this view).

t geoip.location -79.3849, 43.6529

So, it seems like my mapping isn't working. I can confirm the index_patterns in the template matches, so I'm not sure what the problem is...

Regards,

David

2

Shouldn't that be [geoip][location][lat] and [geoip][location][lon]?

Just found an interesting anomaly in logstash configuration parsing. Using arrays

 mutate { add_field => [ "[geoip][location]" , "foo" ] add_field => [ "[geoip][location]" , "bar" ] }

results in

     "geoip" => {
    "location" => [
        [0] "foo",
        [1] "bar"
    ]
},

Whereas using hashes

 mutate { add_field => { "[geoip][location]" => "foo" } add_field => { "[geoip][location]" => "bar" } }

results in

     "geoip" => {
    "location" => "bar"
},

That will come in useful one day :smiley:

3

Hi,

yes I can do it this way too:

#Logstash pipeline

  mutate {
    add_field => [ "[geoip][location][longitude]", "-79.3849" ]
    add_field => [ "[geoip][location][latitude]", "43.6529" ]
    }

I refreshed the index pattern, as well as the actual indices. Still see these as strings though. Shouldn't they end up as a geo_point type?

t  geoip.location.latitude            43.6529
t  geoip.location.longitude       -79.3849
4

oh well no, they are float types.

But location itself "should" be....

5

A geoip has latitude and longitude fields in it, but a geo_point (i.e. [geoip][location]) has lat and lon fields in it.

It does not matter whether you convert them to float or not.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.