# Maps are not showing data if logstash is used **URL:** \ **Category:** Logstash\ **Tags:** maps\ **Created:** [May 21, 2020, 6:48am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679 "2020-05-21T06:48:28Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 21, 2020, 6:48am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/1 "2020-05-21T06:48:28Z")
When i directly send data to Elasticsearch i am able to see the maps data in SIEM and Netflow geolocations. But if i send data through the Logstash i am missing data in SIEM maps and Netflow geoip locations. To get data in maps when i used logstash, what option/plugin/configuration should I used? Can you please refer any any document/blog/configuration/post ? ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 21, 2020, 6:54am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/2 "2020-05-21T06:54:06Z")
this is probably a better question to ask at SIEM forum but here’s a couple of pointer 1. SIEM app uses predefined index pattern. filebeat-_, winlogbeat-_ are some of them, but logstash-\* is not one of them. 2. if you ship logstash with default config, it won’t be included in SIEM app as it will use logstash-\* as index pattern, hence it won’t show in the SIEM network map. 3. for starters , add logstash-\* index pattern to SIEM . [here’s](https://www.elastic.co/guide/en/siem/guide/current/siem-ui-overview.html) a reference on SIEM. 4. also you need geoip plugin filter to do the geoip enrichment to be able to visualize logstash ingested logs in maps. see [here](https://www.elastic.co/guide/en/logstash/current/plugins-filters-geoip.html#plugins-filters-geoip-common-options) for geoip documentation ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 21, 2020, 7:03am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/3 "2020-05-21T07:03:03Z")
Thank you Ptamba for prompt reply. I tried by adding filebeat index(through logstash) pattern in SIEM settings, still not found the data in SIEM maps. I will try Geo Plugin for Logstash and get back to you? Thank you. ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 25, 2020, 4:36am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/4 "2020-05-25T04:36:33Z")
I tried below configuration but destination.geo.location data not found in kibanadiscovery. i am generating data using filebeat netflow module. I am seeing field "destination.ip" in kibana discovery with ip address and destination.geo.location is empty. But if i directly sent the data elasticsearch insetaed of logstash i am seeing destination.geo.location data in kibana discvery and maps. In index patter i am seeing destination.ip is IP. below is the default geo database found logstash logs. Using geoip database {:path=\>"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-6.0.3-java/vendor/GeoLite2-City.mmdb"}. Can you please help me to get geo location data? ```auto input { beats { port => 5045 } } filter { mutate { rename => { "host" => "hostnetflow"}} geoip { source => "destination.ip" } } output { elasticsearch { hosts => ["http://10.252.10.76:9200"] index => "%{[@metadata][beat]}-%{[@metadata][version]}" } } ~~~~~~~~~~~~~~~~~~~~~~~~ ``` ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 25, 2020, 5:21am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/5 "2020-05-25T05:21:31Z")
if you want it to be in the destination.geo then you should update your geoip filter to ``` geoip { source => "destination.ip" target = > “[destination][geo]” } ``` if you don’t specify the target, by default the geo information will be placed under geoip. in your current setup, assuming you have no geo parser failure, you should have geoip.location populated ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 25, 2020, 5:31am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/6 "2020-05-25T05:31:31Z")
> [@ptamba](#): > > `target = > “[destination][geo]”` I updated the config as below, still no destination.geo.location data in kibana discvoery. ```auto geoip { source => "destination.ip" target => "[destination][geo]" tag_on_failure => "_geoip_lookup_failure" } } output { elasticsearch { hosts => ["http://10.252.10.76:9200"] index => "%{[@metadata][beat]}-%{[@metadata][version]}" } } ``` ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 25, 2020, 5:34am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/7 "2020-05-25T05:34:40Z")
can you change the output to stdout and show the output generated by logstash? also are you using role based access control? does your logstash user has privileges to write to filebeat-\* indices? ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 25, 2020, 6:03am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/8 "2020-05-25T06:03:46Z")
Yes i have all permissions. below is the output. ````````````````````````````````````````````auto { "service" => { "type" => "netflow" }, "input" => { "type" => "netflow" }, "source" => { "port" => 53, "ip" => "8.8.8.8", "locality" => "public", "packets" => 2, "bytes" => 346 }, "fileset" => { "name" => "log" }, "netflow" => { "post_nat_destination_ipv4_address" => "xx.xx.xx.xx", "protocol_identifier" => 17, "source_ipv4_address" => "8.8.8.8", "destination_transport_port" => 57272, "post_ip_diff_serv_code_point" => 255, "application_id" => [ [0] 20, [1] 0, [2] 0, [3] 48, [4] 68, [5] 0, [6] 0, [7] 0, [8] 0 ], "ingress_interface" => 3, "flow_start_sys_up_time" => 1206986714, "octet_delta_count" => 346, "post_nat_source_ipv4_address" => "0.0.0.0", "post_octet_delta_count" => 346, "flow_end_reason" => 0, "packet_delta_count" => 2, "post_napt_source_transport_port" => 0, "source_transport_port" => 53, "post_packet_delta_count" => 2, "post_napt_destination_transport_port" => 57272, "exporter" => { "address" => "192.168.252.13:3873", "source_id" => 4, "version" => 9, "timestamp" => "2020-05-25T05:48:23.000Z", "uptime_millis" => 1207211134 }, "egress_interface" => 59, "type" => "netflow_flow", "destination_ipv4_address" => "10.252.10.10", "forwarding_status" => 64, "flow_end_sys_up_time" => 1207029624 }, "agent" => { "type" => "filebeat", "ephemeral_id" => "f1cc8358-cde0-44eb-a6cd-336cc4d33392", "hostname" => "xxxxxxx", "version" => "7.7.0", "id" => "ee22c382-4c82-4026-b738-70164e05b6cf" }, "@version" => "1", "observer" => { "ip" => "192.168.252.13" }, "ecs" => { "version" => "1.5.0" }, "@timestamp" => 2020-05-25T05:48:23.000Z, "flow" => { "id" => "MesTKQVGReI", "locality" => "public" }, "network" => { "community_id" => "1:8f6hNbronq4wEqyCG7ESTdpb8Wk=", "iana_number" => 17, "packets" => 2, "direction" => "unknown", "bytes" => 346, "transport" => "udp" }, "destination" => { "port" => 57272, "ip" => "10.252.10.10", "locality" => "private" }, "event" => { "dataset" => "netflow.log", "kind" => "event", "start" => "2020-05-25T05:44:38.580Z", "module" => "netflow", "created" => "2020-05-25T05:48:23.000Z", "action" => "netflow_flow", "end" => "2020-05-25T05:45:21.490Z", "category" => "network_traffic", "duration" => 42910000000 }, "tags" => [ [0] "ISSQFILE", [1] "INHY", [2] "beats_input_raw_event", [3] "_geoip_lookup_failure" ], "hostnetflow" => { "os" => { "version" => "8 (Core)", "kernel" => "4.18.0-147.8.1.el8_1.x86_64", "codename" => "Core", "name" => "CentOS Linux", "platform" => "centos", "family" => "redhat" }, "name" => "xxxxxxx", "architecture" => "x86_64", "ip" => [ [0] "10.252.10.75", [1] "fe80::d4ee:d927:5185:8d0" ], "mac" => [ [0] "00:15:5d:10:0b:62" ], "hostname" => "xxxxxxx", "id" => "e1cebd3d12bc4510bdecafd61726096c", "containerized" => false } } { "service" => { "type" => "netflow" }, "input" => { "type" => "netflow" }, "source" => { "port" => 58470, "ip" => "10.252.242.5", "locality" => "private", "packets" => 13, "bytes" => 6062 }, "fileset" => { "name" => "log" }, "netflow" => { "post_nat_destination_ipv4_address" => "0.0.0.0", "protocol_identifier" => 6, "application_id" => [ [0] 20, [1] 0, [2] 0, [3] 48, [4] 68, [5] 0, [6] 0, [7] 0, [8] 0 ], "destination_transport_port" => 443, "post_ip_diff_serv_code_point" => 255, "source_ipv4_address" => "10.252.242.5", "ingress_interface" => 59, "flow_start_sys_up_time" => 1207213404, "octet_delta_count" => 6062, "post_nat_source_ipv4_address" => "xx.xx.xx.xx", "post_octet_delta_count" => 6062, "flow_end_reason" => 3, "packet_delta_count" => 13, "post_napt_source_transport_port" => 58470, "egress_interface" => 3, "post_napt_destination_transport_port" => 0, "post_packet_delta_count" => 13, "source_transport_port" => 58470, "type" => "netflow_flow", "exporter" => { "address" => "192.168.252.13:3873", "source_id" => 4, "version" => 9, "timestamp" => "2020-05-25T05:48:28.000Z", "uptime_millis" => 1207216274 }, "destination_ipv4_address" => "138.91.140.216", "forwarding_status" => 64, "flow_end_sys_up_time" => 1207215234 }, "agent" => { "type" => "filebeat", "ephemeral_id" => "f1cc8358-cde0-44eb-a6cd-336cc4d33392", "hostname" => "xxxxxxx", "version" => "7.7.0", "id" => "ee22c382-4c82-4026-b738-70164e05b6cf" }, "@version" => "1", "observer" => { "ip" => "192.168.252.13" }, "ecs" => { "version" => "1.5.0" }, "@timestamp" => 2020-05-25T05:48:28.000Z, "flow" => { "id" => "hY_1pkddS_o", "locality" => "public" }, "network" => { "iana_number" => 6, "community_id" => "1:HyX3xWIqJYOix5Ha7Kwd1nJnTZA=", "packets" => 13, "direction" => "unknown", "bytes" => 6062, "transport" => "tcp" }, "event" => { "dataset" => "netflow.log", "kind" => "event", "module" => "netflow", "start" => "2020-05-25T05:48:25.130Z", "created" => "2020-05-25T05:48:28.000Z", "action" => "netflow_flow", "end" => "2020-05-25T05:48:26.960Z", "category" => "network_traffic", "duration" => 1830000000 }, "destination" => { "port" => 443, "ip" => "138.91.140.216", "locality" => "public" }, "tags" => [ [0] "ISSQFILE", [1] "INHY", [2] "beats_input_raw_event", [3] "_geoip_lookup_failure" ], "hostnetflow" => { "os" => { "version" => "8 (Core)", "kernel" => "4.18.0-147.8.1.el8_1.x86_64", "codename" => "Core", "name" => "CentOS Linux", "platform" => "centos", "family" => "redhat" }, "name" => "xxxxxxx", "architecture" => "x86_64", "ip" => [ [0] "10.252.10.75", [1] "fe80::d4ee:d927:5185:8d0" ], "hostname" => "xxxxxxx", "mac" => [ [0] "00:15:5d:10:0b:62" ], "id" => "e1cebd3d12bc4510bdecafd61726096c", "containerized" => false } } ``````````````````````````````````````````` ```````````````````````````````````````````` ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 25, 2020, 6:15am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/9 "2020-05-25T06:15:07Z")
> [@PraveenKT](#): > > ```auto > "destination" => { > "port" => 443, > "ip" => "138.91.140.216", > "locality" => "public" > }, > "tags" => [ > [0] "ISSQFILE", > [1] "INHY", > [2] "beats_input_raw_event", > [3] "_geoip_lookup_failure" > ], > > ``` as you can see here, the destination object is not populated with geo information and there's a lookup-failure tag. there should be an error related to geoip filter in the logstash log. you might want to update to this: ``` geoip { source => "[destination][ip]" target = > “[destination][geo]” } ``` ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 25, 2020, 6:51am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/10 "2020-05-25T06:51:31Z")
Thank you Ptamba. I am seeing source and destiation locations on Maps. I need another option how do i get connecting lines between source and destianation on Maps? ---
**Author:** ![ptamba](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@ptamba](https://discuss.elastic.co/u/ptamba)\ **Post date:** [May 25, 2020, 7:11am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/11 "2020-05-25T07:11:06Z")
Kibana forum will probably be a more appropriate location to ask for visualization related questions 😊 ---
**Author:** ![jsanz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsanz/32/53734_2.png) [@jsanz](https://discuss.elastic.co/u/jsanz)\ **Post date:** [May 26, 2020, 10:03am UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/12 "2020-05-26T10:03:51Z")
👋 In Elastic Maps you have to use the `Point to Point` layer type to draw lines that connect points on your documents. Here a quick video using Kibana Sample Flights dataset. ![Peek 2020-05-26 12-02](https://us1.discourse-cdn.com/elastic/original/3X/a/4/a43f4977cfd361301f698a9b3926d0da5fc6ea10.gif) ---
**Author:** ![PraveenKT](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praveenkt/32/35483_2.png) [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Post date:** [May 26, 2020, 1:48pm UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/13 "2020-05-26T13:48:17Z")
Thank you Jsanz ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 23, 2020, 1:48pm UTC](https://discuss.elastic.co/t/maps-are-not-showing-data-if-logstash-is-used/233679/14 "2020-06-23T13:48:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.