# UK Post Code to geo data in Logstash? **URL:** \ **Category:** Logstash\ **Created:** [March 2, 2016, 8:16am UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210 "2016-03-02T08:16:24Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![LeeSyd](https://avatars.discourse-cdn.com/v4/letter/l/8dc957/32.png) [@LeeSyd](https://discuss.elastic.co/u/LeeSyd)\ **Post date:** [March 2, 2016, 8:16am UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/1 "2016-03-02T08:16:24Z")
I'm looking for a way to add Geo location data to documents based on UK post code (no IP data, address data only). I've searched but have been unable to find any results and have to admit that I am not a developer so am not sure what approach to take. It looks like the [postcodes.io](http://postcodes.io) api would provide the info I require: - [http://postcodes.io/docs](http://postcodes.io/docs) Am running latest versions of Elasticsearch (2.2), Logstash (2.2) and Kibana (4.4) as of today. If anybody has post code to geo mapping working then some guidance would be much appreciated. Thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 2, 2016, 8:23am UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/2 "2016-03-02T08:23:12Z")
If you can obtain a full dump of the data you could store it in a local file and use the translate filter to map your postcodes to a geolocation. If you really need to make the API lookups you'll probably have to write a custom plugin. It should cache the results to not slam the API endpoint with too many requests. I don't know about a general REST lookup plugin that would do this for you. ---
**Author:** ![LeeSyd](https://avatars.discourse-cdn.com/v4/letter/l/8dc957/32.png) [@LeeSyd](https://discuss.elastic.co/u/LeeSyd)\ **Post date:** [March 7, 2016, 12:46pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/3 "2016-03-07T12:46:04Z")
Thanks @magnusbaeck. I've obtained a dump of the data and can convert to required format but am struggling to get the translate filter to read the dictionary file. To test I have created a small file: - PO11AA:50.799149,-1.0884009 PO11AN:50.799961,-1.079573 PO11AQ:50.799167,-1.0908128 and am using Logstash filter: - ``` translate { dictionary_path => "/etc/logstash/POLatLong.yaml" field => "hl7PostCode" destination => "[geoip][location]" } ``` but get error: - " LogStash::Filters::Translate: Bad Syntax in dictionary file /etc/logstash/POLatLong.yaml" I've tried placing double quotes around each side of the : but with no luck. I'm sure it's a simple one but any pointers? Thanks, Lee ---
**Author:** ![LeeSyd](https://avatars.discourse-cdn.com/v4/letter/l/8dc957/32.png) [@LeeSyd](https://discuss.elastic.co/u/LeeSyd)\ **Post date:** [March 7, 2016, 12:51pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/4 "2016-03-07T12:51:46Z")
Scratch that - just found another post and spotted that I needed a space after the ": " field delimiter in the .yaml file. Have added that and now working OK. Thanks for your assistance @magnusbaeck ---
**Author:** ![Simon\_Thorley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thorley/32/1010_2.png) [@Simon\_Thorley](https://discuss.elastic.co/u/Simon_Thorley)\ **Post date:** [July 15, 2016, 3:54pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/5 "2016-07-15T15:54:27Z")
Hi all, Did you get this functioning as i have ended up at the same point with a yaml file generated from National Statistic data (2.5million lines) and it just hangs. Here is a sample: --- AB10AA: 57.101474,-2.242851 AB10AB: 57.102554,-2.246308 AB10AD: 57.100556,-2.248342 If i use a small set (500 lines) it works fine. Does this plugin have a line limit? Thanks, Simon ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 15, 2016, 4:07pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/6 "2016-07-15T16:07:52Z")
The YAML parser might be running out of memory. CSV is probably cheaper. You'll probably need to extend Logstash's heap size. ---
**Author:** ![Simon\_Thorley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thorley/32/1010_2.png) [@Simon\_Thorley](https://discuss.elastic.co/u/Simon_Thorley)\ **Post date:** [July 15, 2016, 4:15pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/7 "2016-07-15T16:15:14Z")
Can you use CSV for a data lookup against an external source to populate a field? ---
**Author:** ![Simon\_Thorley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simon_thorley/32/1010_2.png) [@Simon\_Thorley](https://discuss.elastic.co/u/Simon_Thorley)\ **Post date:** [July 15, 2016, 4:24pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/8 "2016-07-15T16:24:44Z")
Aha! > [root@simont-fedvm ls-postcode-geoip]# **LS\_HEAP\_SIZE="4g"** /opt/logstash/bin/logstash -f /root/ls-postcode-geoip/ls-test-translate.conf > Settings: Default pipeline workers: 4 > Pipeline main started > BD20 0NZ > { > "message" =\> "BD200NZ", > "@version" =\> "1", > "@timestamp" =\> "2016-07-15T16:18:29.047Z", > "host" =\> "simont-fedvm", > "long-lat" =\> "53.913991,-1.931380" > } All working, cheers. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 15, 2016, 6:13pm UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/9 "2016-07-15T18:13:20Z")
> Can you use CSV for a data lookup against an external source to populate a field? How do you mean? CSV is a flat text file. Unfortunately the translate filter doesn't support lookups again e.g. databases. ---
**Author:** ![jarlrmai](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarlrmai/32/11574_2.png) [@jarlrmai](https://discuss.elastic.co/u/jarlrmai)\ **Post date:** [August 25, 2016, 8:13am UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/10 "2016-08-25T08:13:08Z")
Hi Lee, Found this thread cos I'd doing the same thing, then noticed your field name had "hl7" I'm working using ELK for storing HL7 and just wondered what ideas you had? and how you were getting HL7 into elasticsearch? Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:41am UTC](https://discuss.elastic.co/t/uk-post-code-to-geo-data-in-logstash/43210/11 "2017-07-06T04:41:40Z")