Can't show geopoint in a tile map

Elastic Stack Logstash
1

Hello,

I'm using logstash 2.3.1, elasticsearch 2.3.1 and kibi 0.3.2. I have problems visualizing locations in a map with kibi.

I have the following configuration in logstash:

input {
file {
path => "/opt/logstash-2.3.1/logTest/Dades.csv"
type => "Dades"
start_position => "beginning"
}
}

filter {
csv {
columns => ["c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10", "c11", "c12", "c13", "c14", "c15", "c16", "c17", "c18", "c19", "c20", "c21", "c22", "c23"]
separator => ";"
}

ruby {
code => "
temp = event['c17']
event['c17'] = temp[0..1].to_f+ (temp[2..8].to_f/60)
temp = event['c19']
event['c19'] = temp[0..2].to_f+ (temp[3..8].to_f/60)
"
}
mutate {
convert => {
"c3" => "float"
"c5" => "float"
"c7" => "float"
"c9" => "float"
"c11" => "float"
"c13" => "float"
"c15" => "float"
"c21" => "float"
"c23" => "float"
}

}
date {
match => [ "c1", "dd/MM/YYYY HH:mm:ss.SSS", "ISO8601"]
target => "ts_date"
}

mutate {
rename => [ "c17", "[location][lat]",
"c19", "[location][lon]" ]
}
}

output {
elasticsearch {
hosts => localhost
index => "tram3"
manage_template => false
template => "tram3_template.json"
template_name => "tram3"
template_overwrite => "true"
}
stdout {
codec => rubydebug
}
}

The mapping configuration file (tram3_template.json) is like this:

{
"template": "tram3",
"order": 1,
"settings": {
"number_of_shards": 1
},
"mappings": {
"tram3": {
"_all": {
"enabled": false
},
"properties": {
"location": {
"type": "geo_point",
"geohash": true,
"geohash_prefix": true
}
}
}
}
}

When I import de csv file to elasticsearch it seems that all works ok. The output is something like this:

{
"message" => "26/02/2016 00:00:22.984;Total;4231.143555;Trac1;26.547932;Trac2;-338.939697;AA1;-364.611511;AA2;3968.135010;Reo1;0.000000;Reo2;0.000000;Latitud;4125.1846;Longitud;00213.5219;Speed;0.000000;CVS;3873.429443;\r",
"@version" => "1",
"@timestamp" => "2016-04-25T14:02:52.901Z",
"path" => "/opt/logstash-2.3.1/logTest/Dades.csv",
"host" => "ubuntu",
"type" => "Dades",
"c1" => "26/02/2016 00:00:22.984",
"c2" => "Total",
"c3" => 4231.143555,
"c4" => "Trac1",
"c5" => 26.547932,
"c6" => "Trac2",
"c7" => -338.939697,
"c8" => "AA1",
"c9" => -364.611511,
"c10" => "AA2",
"c11" => 3968.13501,
"c12" => "Reo1",
"c13" => 0.0,
"c14" => "Reo2",
"c15" => 0.0,
"c16" => "Latitud",
"c18" => "Longitud",
"c20" => "Speed",
"c21" => 0.0,
"c22" => "CVS",
"c23" => 3873.429443,
"column24" => nil,
"ts_date" => "2016-02-25T23:00:22.984Z",
"location" => {
"lat" => 41.41974333333334,
"lon" => 2.22535
}
}

I see the location.geohash string and the location geopoint in the index pattern. But when I try to create a tile map, it doesn't show any result. I have tried to set a large time period, but the problem persist.

2

This looks like a LS issue not an ES one, so moving it over :slight_smile:

What does the location data look like in the document?

3

Thanks for your response @warkolm. It's becouse the location in the document is not in the correct format, is DMS (degrees, minutes, secondes).

4

The solution:

Change the type in the file input as it will be used as the document type and the mapping specifies that the mapping type is named tram3 not Dades:

file {
    path => "/opt/logstash-2.3.1/logTest/Dades.csv"
    type => "tram3"
    start_position => "beginning"
    sincedb_path => "/dev/null"
}

The elasticsearch output should look like below, namely, manage_template should be true and use the full path to your tram3_template.json file (make sure to change /full/path/to with the actual path name).

elasticsearch { 
    hosts => localhost
    index => "tram3"
    manage_template => true
    template => "/full/path/to/tram3_template.json"
    template_name => "tram3"
    template_overwrite => "true"
}