How to use geoip logstash filter?

Elastic Stack Logstash
21

Just now i tried GET /_all/_mapping?pretty command and it seems that filebeat is not mapped correctly. Other beats like heartbeat, metricbeat, packetbeat is mapped well.

Where im stuck is what index name do i need to use? Whether it is filebeat-* or just filebeat because im having the following filebeat indices like:

filebeat-2017.08.04,
filebeat-2017.08.03,
filebeat-2017.08.02,

22

Where im stuck is what index name do i need to use?

You don't need to use any name. What do you want to name your indexes? Pick something that works for you and adjust templates and other configuration to match.

24

Now in filebeat template location field is set to geo_point value @magnusbaeck.
"location":{"latitude":"12.9719258","longitude":"80.2171973"} this is my location field value in my log file. After started file beat witht this location value in my log the latitude field is like "location.latitude" and for longitude like "location.langitude". I mapped location as geoip.location and that is fine and mapped as geo_point. But these two fields are set to string values.

25

Do i need to change anything in latitude and longitude fields or those fields are fine?

26

Show the mapping of the location field so we can see that it really is geo_point.

Secondly, https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html documents the four ways a field can look like for ES to properly handle it as geo_point. Your location value doesn't match any of those formats.

27

Following is how i mapped geo_point to location:
{
"filebeat": {
"order": 0,
"template": "filebeat-*",
"settings": {
"index": {
"number_of_shards": "1"
}
},
"mappings": {
"beat": {
"properties": {
"geoip.location": {
"type": "geo_point"
}
}
}
},
"aliases": {}
}
}

28

I didn't ask for the template, I asked for the actual mappings. If I want to see the template I'll ask for the template.

But since you posted your template I might as well comment on it. A couple of days ago I said this:

  • If geoip.location is the field you’ve mapped as geo_point it’s not surprising that the location field isn’t recognized as geo_point. If you want location to be a geo_point you need to make sure that field is mapped as geo_point.
  • That’s not how the mapping of a nested field is set. Consult the ES documentation for details.

These two comments still apply to your current template.

29

what ES documentation @magnusbaeck?

30

See the example at https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings.

31

"beat": {
"properties": {
"geoip": {
"properties": {
"location": {
"type": "geo_point"

This is my mapping

32

This is how i put mapping:

PUT _template/filebeat?pretty
{
"template": "filebeat-*",
"settings": {
"number_of_shards": 1
},
"mappings": {
"beat": {
"properties": {
"geoip.location": {
"type": "geo_point"
}
}
}
}
}

33

is that right or i just set it as location instead of geoip.location?

34

This is how i put mapping:

PUT _template/filebeat?pretty

That sets the template which may or may not apply to your indexes. That template is still wrong because your field in your documents is named location.

35

But i deleted all my filebeat indexes. After clearing all that i made that mapping and what i put in mapping field is set to filebeat template...

36

Also when i refreshed my index pattern in kibana geoip.location field is set to "geo_point"

37

geoip.location geo_point
location.latitude number
location.longitude number

This is how its showing in kibana

38

Have you tried fixing your template so it maps location as geo_point so the mappings match your data?

Over and out. I can't spend any more time on this matter.

39

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.