Logstash Data not visible on coordinate map

Elastic Stack Logstash
1

I created the .conf file and executed through the log stash. It filters using the geo ip.

It creates the data according to requirement.

When I open Kibana and try to visualise in the coordinate map it is not visible even after choosing location in geopoint. I created the template on the console for the index eve then it is not visible.

When I try to create an index through console and visualise the map then it shows in the map.

I could not figure out the problem

2

Can you share the mappings and structure of the indices you are talking about?

3

Template

PUT /_template/template_name
{
"order": 0,
"index_patterns": "logstash*",
"settings": {},
"mappings": {
"default": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}

conf file

input {
file {

type => "geo_point"

path => "/Users/iiita/elk/example/aa.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}

stdin {}

}
filter {

     csv {

            separator => ","
         columns => ["HostIP"]
     }
     geoip{
         source => "HostIP"

     }

}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => ["logstash-venkat"]
}
stdout{
codec => rubydebug
}
}

4

The geoip plugin stores the location in geoip.location and not just location which what you have specified in your mapping. Get the mapping from the actual index (not the template) and check what the mapping for the relevant fields are.

5

can you tell me the way to write mapping the in actual index. when i write in the index mapping on the console it shows type already there

6

What does the actual mapping look like?

7

PUT logstash-venkat/_mapping/my_type
{
"my_type": {
"properties": {
"geoip.location": {
"type": "geo_point"
}
}
}
}

8

PUT logstash-venkat
{
"mappings": {
"doc": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}

Message

#! Deprecation: [default] mapping is deprecated since it is not useful anymore now that indexes cannot have more than one type
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "logstash-venkat"
}

9

Which version of Elasticsearch are you using?

10

6.3.0

11

That looks potentially incorrect. Set the mapping like this:

PUT logstash-venkat/_mapping/my_type
{
  "properties": {
    "geoip": {
      "properties": {
        "location": {
          "type": "geo_point"
        }
      }
    }
  }
}
12

Following error comes out

{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [logstash-venkat] as the final mapping would have more than 1 type: [logs, my_type]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [logstash-venkat] as the final mapping would have more than 1 type: [logs, my_type]"
},
"status": 400
}

13

Yes, you can not change the mapping of an existing index, so will need to drop and recreate it.

14

PUT logstash-venkat

PUT logstash-venkat/_mapping/my_type
{
"properties": {
"geoip": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}

this works fine

when I ran bin/logstash -f logstash-venkat.conf

this gives the problem.

If I first run bin/logstash -f logstash-venkat.conf
it works fine.

afterwards when I run the second mapping then it shows error.

15

Now I got this error

{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "mapper [geoip.location] of different type, current_type [float], merged_type [geo_point]"
}
],
"type": "illegal_argument_exception",
"reason": "mapper [geoip.location] of different type, current_type [float], merged_type [geo_point]"
},
"status": 400
}

16

Stop indexing, delete and recreate the index with the correct mapping and the restart indexing. Or even better - create an index template with the correct mapping so it is automatically applied when a new index is created.

17

I got it after creating the following template

PUT _template/logstash
{
"index_patterns": "logstash-*",
"settings": {
"number_of_replicas": 1,
"number_of_shards": 2
},
"mappings": {
"log": {
"dynamic": "true",
"properties": {
"geoip": {
"dynamic": true,
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}
}

18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.