# Extract Coordinates From Field "log" and Create A New GeoPoint type from it

**URL:** <https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187>\
**Category:** Logstash\
**Created:** [March 30, 2018, 5:59am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187 "2018-03-30T05:59:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kai\_socrates](https://avatars.discourse-cdn.com/v4/letter/k/7ab992/32.png) [@kai\_socrates](https://discuss.elastic.co/u/kai_socrates)\
**Post date:** [March 30, 2018, 5:59am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187/1 "2018-03-30T05:59:40Z")

</div>

I have existing logs which are being added to elasticsearch as a single field called, 'log'. Sample of log content is something like below:  
**[2018-03-28T14:20:16.994Z] "GET /places/v1/radius?maxresults=25&offset=0&locale=eng&lat=47.164444&long=13.571355&radius=700 HTTP/1.1" 200 - 0 12 560 557 "10.240.0.4" "Apache-HttpClient/4.5.3 (Java/1.8.0\_111)"**

Is there any way I can extract the lat and long from the string from the existing data in elasticsearch and convert to new geopoint field and visualize in kibana?

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 31, 2018, 1:20am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187/2 "2018-03-31T01:20:45Z")

</div>

You could create a new pipeline that reads from elasticsearch using the [Elasticsearch Input](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-elasticsearch.html), extracts the location using either the [Grok Filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) or the [KV Filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html), and then _updates_ the existing documents in Elasticsearch using the [Elasticsearch Output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html)

If your query for the input only selected documents that did _not_ have a value for the lat/lon present as a geo-point, each time the pipeline ran it would avoid repeating work.

---

<div class="post-metadata">

**Author:** ![kai\_socrates](https://avatars.discourse-cdn.com/v4/letter/k/7ab992/32.png) [@kai\_socrates](https://discuss.elastic.co/u/kai_socrates)\
**Post date:** [April 2, 2018, 5:03am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187/3 "2018-04-02T05:03:56Z")

</div>

@yaauie Thanks for the suggestion. I've the below config file:

```
input {
  elasticsearch {
    hosts => ["localhost:9200"]
    query => '{ "query": "\"/places/v1\"","analyze_wildcard":true, "sort": ["_doc"] }'
  }
}
 
filter {
 kv {
		field_split => "&?"
		include_keys => ["lat", "long"]
 }
}
 
output {
   
  stdout {
    codec => rubydebug
  }
 
  elasticsearch {
    hosts => ["localhost:9200"]
  }
} 

```

I'm stuck at the output part. How do I map the extracted lat and long fileds to a geo\_point type?

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [April 3, 2018, 6:12am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187/4 "2018-04-03T06:12:34Z")

</div>

You'll need to tell the index that the field is a geo-point, in the mapping for the existing indices, and also in the template so that when a new index is created, it knows how to declare the type.

There are several acceptable forms for the geo-point to take, but the object-form (`{"lat":12.34, "lon":56.78}`) may be easiest to get to from where you're at: by giving the kv filter a [target](https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html#plugins-filters-kv-target), you can have it place the extracted bits together.

You may need to add directives to tell the Elasticsearch output which index to put the document in (there should be an existing `@metadata` field) and what the document's id should be (be careful here -- you'll need to be careful to send the whole document to avoid replacing a complete document with a subset).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2018, 6:12am UTC](https://discuss.elastic.co/t/extract-coordinates-from-field-log-and-create-a-new-geopoint-type-from-it/126187/5 "2018-05-01T06:12:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.