Hi I'm trying to make some map based on some ip address which are private, so I cannot use geoip filter here. Currently I'm implementing some naive method to assign the (lat, lon) coordinates using some if else conditions based on the pattern of sub nets. The problem is after I try to convert the original data to geo point type data, I still get this error:
index pattern does not contain any of the following field types: geo_point
My data flow is filebeat -> logstash -> ES and my logstash config file looks like this:
input {
beats {
port => "5044"
}
}
filter {
json {
source => "message"
}
date {
match => ["timestamp","MMM dd, yyyy hh:mm:ss a"]
target => "@timestamp"
}
ruby {
code => 'event.set("ip2", event.get("ipAddressGuest").split(".")[1])
event.set("ip3", event.get("ipAddressGuest").split(".")[2])'
}
mutate {
convert => { "ip2" => "integer"
"ip3" => "integer"}
}
# More if else statements in the future
if [ip2] == 241 {
ruby {
code => 'event.set("lat", 43.5)
event.set("lon", -80.2)'
}
mutate {
rename => {
'lon' => "[location][lon]"
'lat' => "[location][lat]"
}
}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
index => 'ip_name-%{+YYYY.MM.dd}'
}
}
I look at some of the previous discussions regarding this and I think I might I need to create a template where i assign geopoint data type to location. So I make my own template file like this and add that to the output part of my logstash config file but it still won't work.
{
"template" : "logstash-",
"version" : 50001,
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "norms" : false},
"dynamic_templates" : [ {
"message_field" : {
"path_match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text",
"norms" : false
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "text", "norms" : false,
"fields" : {
"keyword" : { "type": "keyword" }
}
}
}
} ],
"properties" : {
"@timestamp": { "type": "date", "include_in_all": false },
"@version": { "type": "keyword", "include_in_all": false },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "half_float" },
"longitude" : { "type" : "half_float" }
}
},
"location" : {"type": "geo_point"}
}
}
}
}
Can anybody help me with this? Really appreciate it.