Unable to add geo_point mapping with sub lat/lon properties

[r0bot85]

Hello,
Hopefully this is an easy answer but not a stupid question. I am fairly new to the Elastick Stack in general but learning my way around.

I am attempting to get a Coordinate Map visualization using the GeoIP data from RDP logins. I have the winlogbeat configured to send the data to logstash and logstash configured to do the geoip lookup on the source ip which is all working. I got that far following the guide here:

Monitoring Windows Logons with Winlogbeat

However the Index Pattern does not contain a geo_point field and only contains the latitude and longitude in 4 "number" type fields

geoip.latitude|number
geoip.location.lat|number
geoip.location.lon|number
geoip.longitude|number

From the guide and from what i have read and determined i need to setup a new dymanic mapping called geoip.location of the geo_point type containing the sub fields (2 of the ones above) but i have been unable to do so. I found some code below which seems close but fails.

PUT _template/winlogbeat_1
{
"order": 1,
"template": "winlogbeat-*",
"mappings": {
"default": {
"properties": {
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "float" },
"longitude" : { "type" : "float" }
}
}
}
}
}
}

Thanks,
Daryl

[spinscale (Alexander Reelsen)]

can you explain exactly what you refer to when it 'fails'? in your above post? Do you get an exception? Do you get an error when applying the template or do you get an error when indexing the document?

Also note that creating a template does not mean it is applied immediately. You have to create a new index first in order for this to be applied.

[r0bot85]

Of source I will explain more sorry, when I made the post I was 3 hours into trying to figure this out and getting frustrated lol.

When i attempt to run that code to create the template via the console I get the following response.

Here is how i configured the logstash.conf, it was slightly different than the walkthrough but needed as per the JSON field I had coming in that contained the source IP.

Here you can see the geoip data that logstash is bringing into the JSON for the logs.

[spinscale (Alexander Reelsen)]

can you please paste the snippets using markdown, the images are super hard to read and search and grep through. Thanks.

What Elasticsearch version are you using? Remember that from version 7 onwards there are no types anymore and make sure you pick the right version in the documentation as well.

[r0bot85]

Sorry about that, appreciate your help Alexander.

I am on version Kibana 7.3.0 and the latest winlogbeat/logstash from about 6 days ago.

logstash.conf

input {
  beats {
   port => 5044
   type => "log"
  }
}

filter {
  geoip {
    source => "[source][ip]"
  }
}

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{+yyyy.ww}"
    document_type => "%{[@metadata][type]}"
  }
}

Current index pattern is "winlogbeat-*"

Error message when running the PUT _template command

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Root mapping definition has unsupported parameters:  [_default_ : {properties={geoip={dynamic=true, properties={ip={type=ip}, latitude={type=float}, location={type=geo_point}, longitude={type=float}}}}}]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [_default_ : {properties={geoip={dynamic=true, properties={ip={type=ip}, latitude={type=float}, location={type=geo_point}, longitude={type=float}}}}}]",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "Root mapping definition has unsupported parameters:  [_default_ : {properties={geoip={dynamic=true, properties={ip={type=ip}, latitude={type=float}, location={type=geo_point}, longitude={type=float}}}}}]"
    }
  },
  "status": 400
}

As far as the statement about there not being a type i dont understand, because when i go to make a new Coordinate Map visulization it says

The index pattern winlogbeat-* does not contain any of the following compatible field types: geo_point

[spinscale (Alexander Reelsen)]

I can't see the template you are trying to put, but try to omit the _doc part of the template and see if that works. If not, please paste the full command of putting the template here as well in a snippet.

[r0bot85]

It is the one from the article, don't think it has a _doc part that i see

PUT _template/winlogbeat_1
{
  "order": 1,
  "template": "winlogbeat-*",
  "mappings": {
    "_default_": {
      "properties": {
        "geoip"  : {
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip" },
            "location" : { "type" : "geo_point" },
            "latitude" : { "type" : "float" },
            "longitude" : { "type" : "float" }
          }
        }
      }
    }
  }
}

[spinscale (Alexander Reelsen)]

I think we're back to good now, took some time to properly refresh...

[r0bot85]

Alexander,
I am not sure what you mean, was the guide updated? The commands in the guide look the same so i dont know what may have changed or "refreshed".

Thanks

[spinscale (Alexander Reelsen)]

sorry, this reply was not meant for this thread...

try to remove the _default_ field from the above JSON