# How to set up Filebeat to send geoip-info to our hosted instance? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 4, 2021, 6:03pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117 "2021-01-04T18:03:15Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Post date:** [January 4, 2021, 6:03pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/1 "2021-01-04T18:03:15Z")
I am in the process of creating a POC presentation for my management team hoping to show the benefits and advantages of this approach as compared to what we are currently using which is a combination of AppDynamics, the Elastic Stack, AWS Monitoring and Uptrends. What I hope to show is how we can use the Elastic stack as a single point for the monitoring and analysis of our infrastructure. One of the items that I am having issues with is getting Filebeat to send geoip-info to the hosted environment. In our current on premise solution it's easy enough to set the pipeline property to 'geoip-info' as part of the Elasticsearch output configuration but since we're pointing to our hosted environment everything in that area is being ignored. So my question is how do I pass that geoip-info from our local filebeat install to the hosted environment? Thanks, Bill ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 5, 2021, 2:54am UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/2 "2021-01-05T02:54:34Z")
> [@byoungman](#): > > but since we're pointing to our hosted environment everything in that area is being ignored Not sure I follow here. Why does that mean it's being ignored? What is different with the config? ---
**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Post date:** [January 5, 2021, 3:24pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/3 "2021-01-05T15:24:06Z")
I thing I misspoke here are a couple of excerpts from our filebeat.yml file -- #============================= Elastic Cloud ================================== #== These settings simplify using filebeat with the Elastic Cloud ([https://cloud.elastic.co/](https://cloud.elastic.co/)). #== The cloud.id setting overwrites the `output.elasticsearch.hosts` and #== `setup.kibana.host` options. #== You can find the `cloud.id` in the Elastic Cloud web UI. cloud.id: _xxxx_ #== The cloud.auth setting overwrites the `output.elasticsearch.username` and #== `output.elasticsearch.password` settings. The format is `:`. cloud.auth: _xxxx_ #================================ Outputs ===================================== #== Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------ output.elasticsearch: #== Array of hosts to connect to. hosts: ["[http://localhost:9200/](http://localhost:9200/)"] **pipeline: geoip-info** index: "filebeat-%{[fields.doc\_type]}-%{+yyyy.MM.dd}" setup.template.name: xxxx setup.template.pattern: xxxx setup.template.name: xxxx setup.template.pattern: xxxx After posting this I was reading further on the topic and realized that the only thing that gets ignored is the 'hosts' property. I've defined the geoip-info processer with this following the process outlined in [https://www.elastic.co/guide/en/beats/filebeat/7.11/filebeat-geoip.html](https://www.elastic.co/guide/en/beats/filebeat/7.11/filebeat-geoip.html) ``` PUT _ingest/pipeline/geoip-info { "description": "Add geoip info", "processors": [ { "geoip": { "field": "client.ip", "target_field": "client.geo", "ignore_missing": true } }, { "geoip": { "field": "source.ip", "target_field": "source.geo", "ignore_missing": true } }, { "geoip": { "field": "destination.ip", "target_field": "destination.geo", "ignore_missing": true } }, { "geoip": { "field": "server.ip", "target_field": "server.geo", "ignore_missing": true } }, { "geoip": { "field": "host.ip", "target_field": "host.geo", "ignore_missing": true } } ] } ``` I'm still not seeing any geoip info being sent through from filebeat Thanks, Bill ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 5, 2021, 7:18pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/4 "2021-01-05T19:18:05Z")
> [@byoungman](#): > > After posting this I was reading further on the topic and realized that the only thing that gets ignored is the 'hosts' property. Likely because of; > [@byoungman](#): > > #== The cloud.id setting overwrites the `output.elasticsearch.hosts` and > #== `setup.kibana.host` options. And it looks like you have the `cloud.id` set. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 5, 2021, 8:23pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/5 "2021-01-05T20:23:21Z")
Couple things... it is hard to tell but is your yml properly formatted (in the future please format yml code with the `` button. Are the settings under `output.elasticsearch` properly formatted? Is the expected index name being used? Are the documents showing up at all or just not the geoip fields? Did you create your own mapping? with the correct mapping types? ---
**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Post date:** [January 5, 2021, 8:59pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/6 "2021-01-05T20:59:37Z")
Yes the yml is formatted correctly and I'll remember that hint for the future. I actually took a filebeat.yml file from our on premise installation which of course doesn't have the cloud section filled out. The on premise configuration is working of course. The transaction payloads are being sent from filebeat are being sent in to hosted elasticsearch but with out the geoip-info information included. Thanks, Bill ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 5, 2021, 9:08pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/7 "2021-01-05T21:08:53Z")
One thing you can do ... is just get the elasticsearch endpoint from the cloud console and just fill it in with the username and password under the `output.elasticsearch` section just like you would on prem. I suspect there is something simple missed. Don't forget to comment out the `cloud.id` and `cloud.auth` again It will look something like this `https://1234564789cb3bd74c467.us-west1.gcp.cloud.es.io:9243` You can take off the port and it will automatically use 443. that was you can configure it exactly like on prem... give it a try and let us know. Curious What Version you are using? ![Screen Shot 2021-01-05 at 1.01.33 PM](https://us1.discourse-cdn.com/elastic/original/3X/1/9/19f2b7de2897603d2964af10d67c6902cf7df574.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 5, 2021, 9:24pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/8 "2021-01-05T21:24:30Z")
One other thought is to actually remove the `hosts: ["http://localhost:9200/"]` line when using `cloud.id` I have beats using pipelines with `cloud.id` but I don't leave that line it, perhaps there is a bug with that. ---
**Author:** ![byoungman](https://avatars.discourse-cdn.com/v4/letter/b/838e76/32.png) [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Post date:** [January 14, 2021, 5:54pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/9 "2021-01-14T17:54:15Z")
Sorry for the delay in responding but as part of Production Support I often get called off on other issues. So for your first recommendation I commented out the cloud.id and cloud.auth lines and configured the elastic.output host value to be that of my Elastic endpoint and started Filebeat. I saw an error where it couldn't connect to the endpoint - access denied. So I uncommented out the two cloud values from above and restarted filebeat and this time the connection was successfully made but I wasn't seeing any transactions in the Discover section. Going into the Metrics section I could see the filebeat server was processing transactions but they weren't getting indexed into Elasticsearch. I changed the host back to [http://localhost:9200](http://localhost:9200) which the architect that I'm working with for a Proof of Concept recommended as he stated all it will do when it hits that line is throw an error that will be ignored anyway. I started Filebeat up again and traffic is being indexed into Elasticsearch again. -Bill ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [January 14, 2021, 8:39pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/10 "2021-01-14T20:39:14Z")
> [@byoungman](#): > > So for your first recommendation I commented out the cloud.id and cloud.auth lines and configured the elastic.output host value to be that of my Elastic endpoint and started Filebeat. I saw an error where it couldn't connect to the endpoint - access denied. Did you add the `username` and `password` fields to `output.elasticsearch` section.... you would have to do that. Not sure I am quite following / what the issue was But sound like you are working... good ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 11, 2021, 10:39pm UTC](https://discuss.elastic.co/t/how-to-set-up-filebeat-to-send-geoip-info-to-our-hosted-instance/260117/11 "2021-02-11T22:39:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.