How to check if the field exists before to define him as geoppoint


I use Filebeats, Logstash and Elasticsearch to parse differents types of logs files ( IIS, Syslog, Apache, ... ) and watch the result with Kibana. I want to add a map on my dashboard so I use geoip to get gps coordinates from an ip address but sometimes (to set an example : with syslog log file) I only have the username and not the ip address. In result only a part of my output data have the geoip field but i want to define geoip.location as a geopoint in my template. Can i just set the type of the field:"location" on geo_point even if the field is not common to all the output ?

Thank you in advance for your answer

Having a template that defines the type of a field does not mean any documents have to contain that field, so you can set the type of an optional field.

Perfect I thank you ! But my problem isn't solved :disappointed_relieved: i asked this question because when i try to convert geoip.location into a geo_point it doesn't work ( i tried to find the issue without sucess). I followed this tutorial (and others )
i actually use geoip, but if i can't see the location datas on a map, use it seems meaningless to me
Do you have any solutions ?

Before applying the geoip filter on the field xyz, check the existence of that field, like:

filter {
  if [xyz] {
    geoip { source => "xyz" }

Thank you but I already did it. I check the existence of the field and his type ( an ip address ) before to apply geoip.
In fact when i set logstash output on stdout{} i get all the information that i want ( like the geoip field ), but when i set logstash output on elacticsearch and use a tempate to define the field as a geo_point, elasticsearch save it as a number and not a geo_point so i can't use the map visualization in kibana. This is my current issue.

Oh got it now.

Can you share the template? Are you sure it's been applied to your index (have you created it BEFORE creating the index)?

Sorry i'm late
In logstash

geoip {
source => "clientip"
target => "geoip"
mutate {
convert => {
"[geoip][location][lat]" => "float"
"[geoip][location][lon]" => "float"

Mutate part is only to be sure that logstash save location as float and not a number
My template:

"index_patterns": ["squid-*"],
"mappings": {
"default": {
"properties": {
"clientip": { "type": "ip" },
"server_ip": { "type": "ip" },
"user": { "type": "keyword" },
"timestamp": { "type": "date" },
"Log_File_Format": { "type": "keyword" },
"Data_Sent": { "type": "number" },
"Data_Received": { "type": "number" },
"HTTP_Result": { "type": "number" },
"bytes": { "type": "number" },
"delay": { "type": "number" },
"facility": { "type": "number" },
"httpversion": { "type": "number" },
"pid": { "type": "number" },
"priority": { "type": "number" },
"response": { "type": "number" },
"windows_response": { "type": "number" },
"geoip": {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location": { "type": "geo_point" }

I tried a lot of things : create a new field and put lat and lon inside , convert all what i can into float type and check the type of all the fields,...
but it doesn't work
I think i will create a new topic because you answered my first question ( and i thank you ) and it's an other problem

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.