# Filebeat and tile map visualization **URL:** \ **Category:** Logstash\ **Created:** [August 25, 2016, 5:56pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969 "2016-08-25T17:56:12Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [August 25, 2016, 5:56pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/1 "2016-08-25T17:56:12Z")
I'm trying to use Filebeat input to create tile maps of http access. The geoip data are captured, but when I try and create a map, I get the following error: No Compatible Fields: The "filebeat-\*" index pattern does not contain any of the following field types: geo\_point Can I correct this? How? Thanks. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [August 26, 2016, 7:19am UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/2 "2016-08-26T07:19:25Z")
How does your log file look like? Filebeat does not do any log processing. If you want to split up your log lines into fields you need to use Logstash. ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [August 26, 2016, 3:28pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/3 "2016-08-26T15:28:29Z")
ruflin, Sorry, I don't understand your response. I do use Logstash to split up my log lines. Here's the filter I'm using to get geoip data: filter { if [type] == "apache-access" { grok { match =\> { "message" =\> "%{COMBINEDAPACHELOG}" } } date { match =\> ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] locale =\> "en" target =\> "@timestamp" } geoip { source =\> "clientip" database =\> "/etc/logstash/GeoLiteCity.dat" target =\> "geoip" add\_field =\> ["[geoip][coordinates]", "%{[geoip][longitude]}" ] add\_field =\> ["[geoip][coordinates]", "%{[geoip][latitude]}" ] add\_tag =\> "geoip" } mutate { convert =\> ["[geoip][coordinates]", "float"] } } In order to keep my beats indices separate (especially Topbeat and Packetbeat), I've set up output like this: output { if [@metadata][beat] { elasticsearch { hosts =\> ["[http://10.0.101.101:9200](http://10.0.101.101:9200)"] manage\_template =\> false index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } } else { elasticsearch { hosts =\> ["10.0.101.101:9200"] } stdout { codec =\> rubydebug } } } That gives me the following indices: logstash-_, filebeat-_, topbeat-_, packetbeat-_, ... . I can see geoip date in various filebeat log captures, as in the following (sorry for the length of this): { "\_index": "filebeat-2016.08.26", "\_type": "apache-access", "\_id": "AVbHcdzSCSz4NTRp6VCx", "\_score": null, "\_source": { "message": "158.130.6.191 - - [26/Aug/2016:11:22:01 -0400] "HEAD /images/ HTTP/1.1" 404 - "-" "Mozilla/5.0 zgrab/0.x"", "@version": "1", "@timestamp": "2016-08-26T15:22:01.000Z", "input\_type": "log", "fields": null, "type": "apache-access", "beat": { "hostname": "[myhost.com](http://myhost.com)", "name": "[myhost.com](http://myhost.com)" }, "source": "/var/log/httpd/access\_log", "offset": 5621849, "count": 1, "host": "myhost", "tags": [ "log", "beats\_input\_codec\_plain\_applied", "geoip" ], "clientip": "x.x.x.x", "ident": "-", "auth": "-", "timestamp": "26/Aug/2016:11:22:01 -0400", "verb": "HEAD", "request": "/images/", "httpversion": "1.1", "response": "404", "referrer": ""-"", "agent": ""Mozilla/5.0 zgrab/0.x"", "geoip": { "ip": "y.y.y.y", "country\_code2": "US", "country\_code3": "USA", "country\_name": "United States", "continent\_code": "NA", "region\_name": "PA", "city\_name": "Philadelphia", "postal\_code": "19104", "latitude": 39.9597, "longitude": -75.1968, "dma\_code": 504, "area\_code": 215, "timezone": "America/New\_York", "real\_region\_name": "Pennsylvania", "location": [ -75.1968, 39.9597 ], "coordinates": [ -75.1968, 39.9597 ] } }, "fields": { "@timestamp": [ 1472224921000 ] }, "highlight": { "tags": [ "@kibana-highlighted-field@geoip@/kibana-highlighted-field@" ] }, "sort": [ 1472224921000 ] } Great. Now how do I visualize this in Kibana? As stated in the OP, I get "No Compatible Fields: The "filebeat-_" index pattern does not contain any of the following field types: geo\_point" if I use the filebeat-_ indices, and no output if I use logstash-\* indices. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 27, 2016, 2:24pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/4 "2016-08-27T14:24:24Z")
I suggest you edit your post and change its category to Logstash since this is a Logstash question. Logstash's default index template which would've taken care of mapping your GeoIP field as geo\_point only applies to logstash-\* indexes. You need to install another index template (or modify the existing template) to also apply to filebeat-\* indexes. ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [August 30, 2016, 8:13pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/5 "2016-08-30T20:13:05Z")
Thanks, Magnus. I used the command "curl -XGET '[http://localhost:9200/\_template/logstash?pretty](http://localhost:9200/_template/logstash?pretty)' \> /tmp/logstash.template", and found this in logstash.template: { "logstash" : { "order" : 0, "template" : "logstash-\*", "settings" : { "index" : { "refresh\_interval" : "5s" } }, If I wanted to add filebeat-\*, would it be: ``` "template" : "logstash-*" "filebeat-*", ``` and, if so, what does that do to my current logstash and filebeat output, if anything? I didn't find a filebeat template, at least not with the curl command above, substituting in "filebeat?". Should there be a filebeat template? If not, how would I create one (copy logstash template, with appropriate changes?), and would it change or break the filebeat output I'm seeing now? Btw, I find in the logstash template: ``` }, { "geo_point_fields" : { "mapping" : { "type" : "geo_point", "doc_values" : true }, "match_mapping_type" : "geo_point", "match" : "*" } } ], "_all" : { "omit_norms" : true, "enabled" : true }, "properties" : { "@timestamp" : { "type" : "date", "doc_values" : true }, "geoip" : { "dynamic" : true, "type" : "object", "properties" : { "ip" : { "type" : "ip", "doc_values" : true }, "latitude" : { "type" : "float", "doc_values" : true }, "location" : { "type" : "geo_point", "doc_values" : true }, "longitude" : { "type" : "float", "doc_values" : true } } }, ``` I guess the geo\_point part is what I'm after. Thank you. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 30, 2016, 8:31pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/6 "2016-08-30T20:31:58Z")
> If I wanted to add filebeat-\*, would it be: > ``` > "template" : "logstash-*" "filebeat-*", > > ``` No, because that's not valid JSON. I don't think a single template can include multiple index name patterns so you'll have to install a separate template. > I didn't find a filebeat template, at least not with the curl command above, substituting in "filebeat?". Should there be a filebeat template? Logstash isn't creating it for you, so no. You'll have to create it yourself. However, Logstash's elasticsearch output can manage it for you. > and would it change or break the filebeat output I'm seeing now? It shouldn't break anything, but fields _will_ be mapped differently so you'll see different behavior in Kibana. > I guess the geo\_point part is what I'm after. Yes. ---
**Author:** ![rsaeks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rsaeks/32/5068_2.png) [@rsaeks](https://discuss.elastic.co/u/rsaeks)\ **Post date:** [September 6, 2016, 8:38pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/7 "2016-09-06T20:38:35Z")
I'm running into the same issue. I'm using a filebeat input and am adding in the geoip location information successfully. When trying to generate the tile map, I receive the message No Compatible Fields: The "filebeat-\*" index pattern does not contain any of the following field types: geo\_point but am unsure how to get this going. Looking at the logstash-\* index geoip.location shows as geopoint but filebeat-\* index shows the same as just number However running: curl -XGET '[http://localhost:9200/\_template/filebeat?pretty](http://localhost:9200/_template/filebeat?pretty)' shows: { "filebeat" : { "order" : 0, "template" : "filebeat-_", "settings" : { "index" : { "refresh\_interval" : "5s" } }, "mappings" : { "default" : { "dynamic\_templates" : [ { "template1" : { "mapping" : { "ignore\_above" : 1024, "index" : "not\_analyzed", "type" : "{dynamic\_type}", "doc\_values" : true }, "match" : "_" } } ], "\_all" : { "norms" : { "enabled" : false }, "enabled" : true }, "properties" : { "@timestamp" : { "type" : "date" }, "geoip" : { "dynamic" : true, "type" : "object", "properties" : { "location" : { "type" : "geo\_point" } } }, "offset" : { "type" : "long", "doc\_values" : "true" }, "message" : { "index" : "analyzed", "type" : "string" } } } }, "aliases" : { } } } I've refreshed the index and well as deleted and re-created but still don't have the ability to use a geohash. I'm sure it is something simple! ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [September 7, 2016, 3:24pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/8 "2016-09-07T15:24:28Z")
If I import filebeat.template.json from a server running Filebeat to my ELK server, edit that file to include the geo\_point part, and load the template with "curl -XPOST '[http://localhost:9200/\_template/filebeat?pretty](http://localhost:9200/_template/filebeat?pretty)' -d @/home/me/filebeat.template.json", will that get the tile maps working with filebeat? If so, what, exactly, do I put in filebeat.template.json to make it so. Thanks. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 8, 2016, 5:31am UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/9 "2016-09-08T05:31:46Z")
The template you posted previously should work with the event you posted even earlier. If it doesn't work, please post the mappings of an actual index (so that we can see that the template has been applied) and an example document from ES that contains the geoip field. ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [September 8, 2016, 5:00pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/10 "2016-09-08T17:00:19Z")
Magnus, Sorry, I'm thick as a brick, and hope I'm not being too exasperating. When you say "The template you posted previously ... ", are you referring to the template that rsaeks actually posted? The only template that is loaded in my ELK is the Logstash template. Don't I need a Filebeat template? Diggy ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 8, 2016, 6:36pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/11 "2016-09-08T18:36:48Z")
Oh, sorry. Yes, you need an index template that matches Filebeat indexes and that maps your `[geoip][coordinates]` field as geo\_point. You can use the Logstash template as a starting point. ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [September 8, 2016, 6:52pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/12 "2016-09-08T18:52:27Z")
OK. However, since I don't have a real good idea as to how to create the proper template, would this work?: ``` { "mappings": { "_default_": { "_all": { "enabled": true, "norms": { "enabled": false } }, { "geo_point_fields" : { "mapping" : { "type" : "geo_point", "doc_values" : true }, "match_mapping_type" : "geo_point", "match" : "*" } } ], "_all" : { "omit_norms" : true, "enabled" : true }, "dynamic_templates": [ { "template1": { "mapping": { "doc_values": true, "ignore_above": 1024, "index": "not_analyzed", "type": "{dynamic_type}" }, "match": "*" } } ], "properties": { "@timestamp": { "type": "date" }, "message": { "type": "string", "index": "analyzed" }, "offset": { "type": "long", "doc_values": "true" } } } }, "settings": { "index.refresh_interval": "5s" }, "template": "filebeat-*" } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 8, 2016, 7:49pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/13 "2016-09-08T19:49:59Z")
Almost. You have a block that says what settings to apply to fields that indeed are mapped as geo\_point, but there's nothing that actually maps a field as geo\_point. You need to add something like ```nohighlight "geoip": { "properties": { "coordinates": { "type": "geo_point" } } } ``` to your current "properties" block if you want `[geoip][coordinates]` to be a geo\_point field. ---
**Author:** ![Diggy](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@Diggy](https://discuss.elastic.co/u/Diggy)\ **Post date:** [September 8, 2016, 10:02pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/14 "2016-09-08T22:02:31Z")
I created this template, and loaded it with "curl -XPOST '[http://local:9200/\_template/filebeat?pretty](http://local:9200/_template/filebeat?pretty)' -d @/home/me/filebeat.template.json" on my ELK server: ``` { "mappings": { "_default_": { "_all": { "enabled": true, "norms": { "enabled": false } }, "dynamic_templates": [ { "template1": { "mapping": { "doc_values": true, "ignore_above": 1024, "index": "not_analyzed", "type": "{dynamic_type}" }, "match": "*" } } ], "properties": { "@timestamp": { "type": "date" }, "message": { "type": "string", "index": "analyzed" }, "offset": { "type": "long", "doc_values": "true" }, "geoip": { "properties": { "coordinates": { "type": "geo_point" } } } } } }, "settings": { "index.refresh_interval": "5s" }, "template": "filebeat-*" } ``` I got: ``` { "acknowledged" : true } ``` in return. However, I still get: "No Compatible Fields: The "filebeat-\*" index pattern does not contain any of the following field types: geo\_point" in Kibana. Arrrgh! Do I still have a misconfiguration? Diggy ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 9, 2016, 5:30am UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/15 "2016-09-09T05:30:54Z")
Since creating the template, did you create a new index and populate it with data? Have you verified that the created index gets the expected mappings? Did you refresh the field list in Kibana? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 15, 2016, 5:56pm UTC](https://discuss.elastic.co/t/filebeat-and-tile-map-visualization/58969/16 "2016-09-15T17:56:17Z")
This topic was automatically closed after 21 days. New replies are no longer allowed.