Geop.location not of geo_pooint. elastic.co 5.6.4 on RPI3

Elastic Stack Elasticsearch
1

Hey, i did some change, renew my conf and my MAP is gone. No field of geo_point type.
I use the geoip pipeline filter and all is running fine except the map.

GET /logstash-2018.05/_mapping return a struct showing that geoip has a field name location with lat and lon. (see capture) they are filled correctly by the filter also.
I read many threads and in one that i could modify the logstash indice and delete the index then i will recreate. I apply the following
PUT _template/logstash
{
"template": "logstash-*",
"settings": {
"number_of_replicas": 1,
"number_of_shards": 1
},
"mappings": {
"syslog": {
"dynamic": "true",
"properties": {
"geoip": {
"dynamic": true,
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}

and then delete the index pattern and recreate again. it failed.

Can someone help ?
thanks

07
Capture d’écran 2018-05-27 à 17.38.07.png2300×1010 343 KB

2

Please don't post images of text as they are hardly readable and not searchable.

Instead paste the text and format it with </> icon. Check the preview window.

Could you share the full mapping?

3

Format error

4

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

5

Format errata

6

Did you check the preview window?

7

Hi, new try.

I try to format it as possible to show the geoip struct and location which should be the issue.
Just to relight the situation : Logstash index does not allow to be mapped as there's no geo_point field. GeoIP information are perfectly filled per the pipeline filter from the VarX vaiable which is the IP detected. It works fine except the map.

here is the GET /logstash-2018.05/_mapping

"logstash-2018.05":
    { "mappings": { 
    "syslog": { "properties": 
    { "@timestamp": { "type": "date" }, "@version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "CPU": { "type": "long" }, "Disk": { "type": "long" }, "Port": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "RAM": { "type": "long" }, "Temp": { "type": "long" }, 
    "Varx": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } },

 "geoip": { "properties": { 
"city_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } },
 "continent_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } },
 "country_iso_code": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, 
"location": { "properties": { "lat": { "type": "float" }, "lon": { "type": "float" } } }, 
"region_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } 
}, 
"host": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "message": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "received_at": { "type": "date" }, "received_from": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_facility": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_facility_code": { "type": "long" }, "syslog_hostname": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_message": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_pid": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_program": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_severity": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "syslog_severity_code": { "type": "long" }, "syslog_timestamp": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "type": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } } }` /&gt;

I try to

 `PUT _template/logstash
{
"template": "logstash-*",
"settings": {
"number_of_replicas": 1,
"number_of_shards": 1
},
"mappings": {
    "syslog": {
    "dynamic": "true",
    "properties": {
    "geoip": {
        "dynamic": true,
        "properties": {
            "location": {
                "type": "geo_point"
            }
        }
     }
}
}
}`

Then delete the index and recreate it.

8

Look

This is not formatted code

This is formatted code

Can you do this please? If not please explain what you don't understand.

9

Oh yes got it now Sorry ! I corrected upper in the thread and removed the unformated messages. Thanks. Don't know how insert colors in the formatted like you did in red.

10

I had to parse your post to make it readable.

{
   "mappings":{
      "syslog":{
         "properties":{
            "@timestamp":{
               "type":"date"
            },
            "@version":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "CPU":{
               "type":"long"
            },
            "Disk":{
               "type":"long"
            },
            "Port":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "RAM":{
               "type":"long"
            },
            "Temp":{
               "type":"long"
            },
            "Varx":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "geoip":{
               "properties":{
                  "city_name":{
                     "type":"text",
                     "fields":{
                        "keyword":{
                           "type":"keyword",
                           "ignore_above":256
                        }
                     }
                  },
                  "continent_name":{
                     "type":"text",
                     "fields":{
                        "keyword":{
                           "type":"keyword",
                           "ignore_above":256
                        }
                     }
                  },
                  "country_iso_code":{
                     "type":"text",
                     "fields":{
                        "keyword":{
                           "type":"keyword",
                           "ignore_above":256
                        }
                     }
                  },
                  "location":{
                     "properties":{
                        "lat":{
                           "type":"float"
                        },
                        "lon":{
                           "type":"float"
                        }
                     }
                  },
                  "region_name":{
                     "type":"text",
                     "fields":{
                        "keyword":{
                           "type":"keyword",
                           "ignore_above":256
                        }
                     }
                  }
               }
            },
            "host":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "message":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "received_at":{
               "type":"date"
            },
            "received_from":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_facility":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_facility_code":{
               "type":"long"
            },
            "syslog_hostname":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_message":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_pid":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_program":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_severity":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "syslog_severity_code":{
               "type":"long"
            },
            "syslog_timestamp":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "tags":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "type":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            }
         }
      }
   }
}

And:

PUT _template/logstash
{
   "template":"logstash-*",
   "settings":{
      "number_of_replicas":1,
      "number_of_shards":1
   },
   "mappings":{
      "syslog":{
         "dynamic":"true",
         "properties":{
            "geoip":{
               "dynamic":true,
               "properties":{
                  "location":{
                     "type":"geo_point"
                  }
               }
            }
         }
      }
   }
}

Just wondering why did you set "dynamic": true, in geoip field?

11

Hi, thanks how come it's nice with colors ? i will practice.
I put the dynamic to true because i found it in many thread on the subject. I have a poor level on Elastisearch but i am having fun. I have not been able to solve.

12

Pasting correct JSON (was missing curly braces or too many ones in your post), nice indentation... That's the key of the success.

I have not been able to solve.

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem.

13

hi,

  1. When I try to create a map of my logstash index I got the error :
    No Compatible Fields: The "logstash-*" index pattern does not contain any of the following field types: geo_point.
  2. I read many treads and found that geo_point is a [lat,long] struct with lat-long declared as numbers but it's not strictly declared as GeoPoint.
  3. The GeoIP pipeline filter is running fine, filling all the field of the mentionned GeoIP structure ;
  4. The thread describe the logstash index struct as well as a PUT procedure to try to register a GeoIP struct. cf. upper ref PUT _template/logstash

I am still unable to have a Map with lat,lon fields filled with correct value. and the message decribes in Step 1.

Regards

14

Please share what you ran.

15

I ran it again to make sure of the process and it's working

  1. Delete from the GUI Managment/ Index Pattern : Delete logstash-*

  2. PUT _template/logstash according to the command line in the thread

  3. Register VarX fied (may be not necessary as it is already in the datas)

     PUT /.kibana/_mapping/syslog
 {
  "properties": {
    "Varx": {
       "type": "text"
    }
   }  
}

  4. I register the GeoIP pipeline filter to VarX variable (containing the IP address)

    PUT _ingest/pipeline/geoip
{
  "description" : "Add geoip info",
 "processors" : [
  {
     "geoip" : {
                "field" : "Varx"
               }
    }]
}

  5. I reconstruct the index from the GUI Managment / Index Pattern : Create Logstash-* and it works :smile:

I am so happy I have the MAP and GeoIP.location is of geo_point type in the Kibana/Managment/Index Pattern as in the screen capture.

24
Capture d’écran 2018-06-03 à 10.08.24.png1194×186 7.83 KB

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.