# Importing csv with geo locations using logstash 5 / kibana 5 **URL:** \ **Category:** Logstash\ **Created:** [December 6, 2016, 8:56pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223 "2016-12-06T20:56:36Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Snyder](https://avatars.discourse-cdn.com/v4/letter/s/f14d63/32.png) [@Snyder](https://discuss.elastic.co/u/Snyder)\ **Post date:** [December 6, 2016, 8:56pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/1 "2016-12-06T20:56:36Z")
Hello, Can someone help me with the configuration of a logstash 5 conf file and/or kibana 5? I'm running logstash, elasticsearch and kibana locally on a mac 10.10.x. The topic is about the combination of loading csv files in combination with geo locations (latitude, longitude). UseCase - import of a given csv file using logstash - the csv is structured lines/columns and includes latitude/longitude geo location - running the below pasted conf-file - exploring the geo-data in a kibana 5 tile-map Results - data are piping into elasticsearch and showing up in kibana - the lat/lon are shown as GeoHASH in the tile-map configuration section in the subsequent dropdown a geoip.longitude is appearing - once picking up the geoip kibana prompts "no results found" Question - could you please double-check my conf file example - do I need to re-config my logstash load file - do I need to configure/set in kibana to get my "location" work Appreciate your help ############################ my\_geo\_config input { stdin { type =\> "stdin-type" } file { path =\> ["/Users/.../Test\_020.csv"] start\_position =\> "beginning" } } filter { csv { columns =\> ["field\_A","field\_B","field\_C\_date","field\_D","latitude","longitude"] separator =\> "," } date { match =\> ["field\_C\_date", "YYYY-MM-DD"] } ``` mutate { convert => ["field_A", "string"] convert => ["field_B", "integer"] convert => ["field_D", "integer"] } mutate { convert => {"latitude" => "float"} convert => {"longitude" => "float"} add_field => ["location", "%{latitude},%{longitude}"] convert => {"location" => "float"} } } ``` output { elasticsearch { hosts =\> ["localhost:9200"] } stdout { codec =\> rubydebug } } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 7, 2016, 12:33am UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/2 "2016-12-07T00:33:51Z")
It looks ok, do you have the right time range in KB? ---
**Author:** ![Snyder](https://avatars.discourse-cdn.com/v4/letter/s/f14d63/32.png) [@Snyder](https://discuss.elastic.co/u/Snyder)\ **Post date:** [December 7, 2016, 9:54pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/3 "2016-12-07T21:54:03Z")
Dear Mark, Thanks for your reply. Could you please provide me with an hint regarding the time range? Regards Marc ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 7, 2016, 10:16pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/4 "2016-12-07T22:16:08Z")
Top right - [https://www.elastic.co/guide/en/kibana/current/set-time-filter.html](https://www.elastic.co/guide/en/kibana/current/set-time-filter.html) ---
**Author:** ![Snyder](https://avatars.discourse-cdn.com/v4/letter/s/f14d63/32.png) [@Snyder](https://discuss.elastic.co/u/Snyder)\ **Post date:** [December 9, 2016, 3:38pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/5 "2016-12-09T15:38:27Z")
Dear Mark (with "k"), Thanks a million. I'm afraid my data conversion is not very properly. I'm trying to figure out that particular item. Never the less I came across another issue. As for configuring the \*.config file I'm dealing with an input csv of about 100 rows. Now when scaling the file to the full size about 500.000 rows the following happens: - logstash is processing until "Successfully started Logstash API endpoint {:port=\>9600}" then nothing else happens - a few lines above logstash prompts " Starting pipeline {"id"=\>"main", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>1000}" - if I exclude the latitude, longitude columns then the file is processed (or sometimes not) Never had that kind of experience with stack #4 Also matching data seems to be tricky. Just want to index the "voucher\_date" (YYYY-MM-DD in the source). So far it doesn't work with date { match =\> ["voucher\_date", "YYYY MM dd"] } Have you or anyone an idea how to design a \*config unsing csv, geo(latitude, longitude) voucher\_date string to date and having more than 100 rows in the source-file? Greatly appreciate your contributions Gruß Marc ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 11, 2016, 1:25am UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/6 "2016-12-11T01:25:20Z")
> [@Snyder](#): > > Now when scaling the file to the full size about 500.000 rows the following happens:- logstash is processing until "Successfully started Logstash API endpoint {:port=\>9600}" then nothing else happens- a few lines above logstash prompts " Starting pipeline {"id"=\>"main", "pipeline.workers"=\>8, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>1000}"- if I exclude the latitude, longitude columns then the file is processed (or sometimes not)Never had that kind of experience with stack #4 Sounds like a sincedb issue more than anything. ---
**Author:** ![Snyder](https://avatars.discourse-cdn.com/v4/letter/s/f14d63/32.png) [@Snyder](https://discuss.elastic.co/u/Snyder)\ **Post date:** [December 11, 2016, 9:44pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/7 "2016-12-11T21:44:33Z")
Dear Mark, Thanks for keeping up with the above. It took a little time to figure it out. Finally I converted the \*csv from UTF8 "Legacy Mac OS (CR)" into "Unix (LF)" it started working. Thanks again and have a great one Marc ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 12, 2016, 12:56am UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/8 "2016-12-12T00:56:13Z")
Ohhh, that is odd. Can you raise something against the GH repo for the plugin? We should really handle this for the user 🙂 ---
**Author:** ![Snyder](https://avatars.discourse-cdn.com/v4/letter/s/f14d63/32.png) [@Snyder](https://discuss.elastic.co/u/Snyder)\ **Post date:** [December 14, 2016, 9:37pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/9 "2016-12-14T21:37:16Z")
Dear Mark, Yep, GH = GitHub, isn’t it? May need a little time, as I’m a bit busy at the moment. Now I’m struggling again with the GeoData. They don’t appearing as geo\_point/geohash in kibana. Referring to the import issue I’m afraid I’m somehow lost. I want to review and adjust my complete workflow end-2-end. Is there a paper/blog/website available which provides an “ES 5 stack csv-geo” breakdown in principal or from bird’s eye perspective? Based on an example or something like this? I’m looking for a kind of an recipe or a handy checklist? Appreciate your help Marc ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 14, 2016, 9:49pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/10 "2016-12-14T21:49:11Z")
> [@Snyder](#): > > Yep, GH = GitHub, isn’t it? Yep! 🙂 > [@Snyder](#): > > Now I’m struggling again with the GeoData. They don’t appearing as geo\_point/geohash in kibana. What do they show as? Check `indexname/_mapping` to see what the field is defined as in ES. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 11, 2017, 9:49pm UTC](https://discuss.elastic.co/t/importing-csv-with-geo-locations-using-logstash-5-kibana-5/68223/11 "2017-01-11T21:49:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.