GEOPOINT not recognized

Elastic Stack Logstash
1

Hi guys, i'm using logstash 6.2.4 and i have some problem to upload my longitude and latitude point in kibana.
I red a lot regarding this issue, but i still have some doubt.

my logstash config file looks like:
input {
file {
path => "......."
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {

csv {
	separator => ";"
	columns => ["cod_ibge_infeccao_7","cod_ibge_residencia","ano","mes","sexo","raca","resexame","faixa_etaria","sintomas","zona","gestante","oport_diag","oport_trat","sistema_not","sum","nome_municipio","UF","Estado","Latitude","Longitude"]

}
mutate {convert => ["cod_ibge_infeccao_7", "integer"] }
mutate {convert => ["cod_ibge_residencia", "integer"] }
mutate {convert => ["ano", "integer"] }
mutate {convert => ["mes", "integer"] }
mutate {convert => ["sexo", "string"] }
mutate {convert => ["raca", "string"] }
mutate {convert => ["resexame", "string"] }
mutate {convert => ["faixa_etaria", "string"] }
mutate {convert => ["sintomas", "string"] }
mutate {convert => ["zona", "integer"] }
mutate {convert => ["oport_diag", "string"] }
mutate {convert => ["sistema_not", "string"] }
mutate {convert => ["sum", "integer"] }
mutate {convert => ["nome_municipio", "string"] }
mutate {convert => ["UF", "string"] }
mutate {convert => ["Estado", "string"] }
mutate {convert => ["Latitude", "float"] }
mutate {convert => ["Longitude", "float"] }
mutate { rename => {"Latitude" => "[location][lat]"} }
	mutate { rename => {"Longitude" => "[location][lon]"} }
mutate { remove_field => [ "host"] }

}

output {
elasticsearch {
template => "....../elasticsearch-template.json"
template_name => "logstash-*"
hosts => "localhost"
index => "base_geo_prova"
document_type => "m_geo"
user => "elastic"
password => "changeme"
sniffing => true
manage_template => true
}
stdout { codec => "rubydebug" }

my template file looks like this one:

{
"template" : "logstash-",
"settings" : {
"index.refresh_interval" : "5s"
},
"mappings" : {
"default" : {
"_all" : {"enabled" : true, "omit_norms" : true},
"dynamic_templates" : [ {
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true,
"fielddata" : { "format" : "disabled" }
}
}
}, {
"string_fields" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string", "index" : "analyzed", "omit_norms" : true,
"fielddata" : { "format" : "disabled" },
"fields" : {
"raw" : {"type": "string", "index" : "not_analyzed", "ignore_above" : 256}
}
}
}
} ],
"properties" : {
"@timestamp": { "type": "date" },
"@version": { "type": "string", "index": "not_analyzed" },
"geoip" : {
"dynamic": true,
"properties" : {
"ip": { "type": "ip" },
"location" : { "type" : "geo_point" },
"latitude" : { "type" : "float" },
"longitude" : { "type" : "float" }
}
},
"location" : { "type": "geo_point" }
}
}
}
}

When i'm trying to push my data into kibana not recognize them like geopoint.
So after reading some post i tryied in DevTools after importing my data to mapping before my points.:

PUT base_geo_prova
{
"mappings": {
"_doc": {
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}

But when i'm trying after that to import my csv tell me that index is already exist and cannot import any data.
Can you help me regarding this issue?

Thanks a lot

Alberto

2

Your template applies to indexes whose name match logstash-* but your index is named base_geo_prova so the template doesn't apply.

3

Thanks a lot, now i don't have anymore message error or stuff but still dosn't convert.
1- step in kibana run in devtools:
PUT _template/base_geo_prova
{
"template": "stations",
"settings": {
"number_of_replicas": 1,
"number_of_shards": 2
},
"mappings": {
"my_type": {
"dynamic": "true",
"properties": {
"geoip": {
"dynamic": true,
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
}
}

second step after changing in my template base_geo_prova i willi ngest my data in kibana.

It takes the datas but without trasforming in geo.point.
my corrections in the output was:
template_name => "base_geo_prova"

and in the template of course: "template" : "base_geo_prova",

but still ingest the datas normally
Any suggestions?

4

Your template should look like this:

{
  "template": "name-of-your-index",
  ...
}

and your elasticsearch output should look like this:

elasticsearch {
  index => "name-of-your-index"
  ...
}

Is that what your configurations look like? I can't tell from what you write above.

5

Thanks a lot. Now it works fine. We can closed this post.
Best Regards

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.