How to map a non-standard geo_type

Elastic Stack Elasticsearch
1

Hi,

I ingest data from CouchDB into ElasticSearch and would like my lat, lon to
be mapped as geo_type, but I don't know how to do it, since my lat, lon is
not fitting any of the geo_type examples from the guide.

My data looks like this:

"location": {
"latitude": 37.632571793,
"name": "Pacifica Pier",
"longitude": -122.494277492,
"id": 5076543
}

(of course there are a bunch of other fields)

Since the "location" doesn't have an array [lat, lon] I'm not sure how to specify the type to be "geo_type".

Would something like this work?

"location":{
         "properties":{
                   "id":{"type":"long"},
                   "latitude":{"type":"geo_type"},
                   "longitude":{"type":"geo_type"},
                   "name":{"type":"string"}

          }

}

How can I combine the two values into one array?

I'd highly appreciate any suggestions or help.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

It can not work out of the box.
You could think of using a script to create a new locationgeo field, and copy lat,lon values in it:
"location" : "lat,lon"
Have a look at: https://github.com/elasticsearch/elasticsearch-river-couchdb#script-filters

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 01:55, Oleg Aulov aulov.oleg@gmail.com a écrit :

Hi,

I ingest data from CouchDB into ElasticSearch and would like my lat, lon to be mapped as geo_type, but I don't know how to do it, since my lat, lon is not fitting any of the geo_type examples from the guide.

My data looks like this:

"location": {
"latitude": 37.632571793,
"name": "Pacifica Pier",
"longitude": -122.494277492,
"id": 5076543
}

(of course there are a bunch of other fields)

Since the "location" doesn't have an array [lat, lon] I'm not sure how to specify the type to be "geo_type".

Would something like this work?

"location":{
         "properties":{
                   "id":{"type":"long"},
                   "latitude":{"type":"geo_type"},
                   "longitude":{"type":"geo_type"},
                   "name":{"type":"string"}
          }
}

How can I combine the two values into one array?

I'd highly appreciate any suggestions or help.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

My answer supposed that you are using couchdb river. Is it true?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 06:51, David Pilato david@pilato.fr a écrit :

It can not work out of the box.
You could think of using a script to create a new locationgeo field, and copy lat,lon values in it:
"location" : "lat,lon"
Have a look at: https://github.com/elasticsearch/elasticsearch-river-couchdb#script-filters

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 01:55, Oleg Aulov aulov.oleg@gmail.com a écrit :

Hi,

I ingest data from CouchDB into ElasticSearch and would like my lat, lon to be mapped as geo_type, but I don't know how to do it, since my lat, lon is not fitting any of the geo_type examples from the guide.

My data looks like this:

"location": {
"latitude": 37.632571793,
"name": "Pacifica Pier",
"longitude": -122.494277492,
"id": 5076543
}

(of course there are a bunch of other fields)

Since the "location" doesn't have an array [lat, lon] I'm not sure how to specify the type to be "geo_type".

Would something like this work?

"location":{
         "properties":{
                   "id":{"type":"long"},
                   "latitude":{"type":"geo_type"},
                   "longitude":{"type":"geo_type"},
                   "name":{"type":"string"}
          }
}

How can I combine the two values into one array?

I'd highly appreciate any suggestions or help.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Hi David, thanks for a prompt reply! Yes, using CouchDB river.

Makes sense. So if there is a script in the river that moves lat, lon into
an array then Elasticsearch is going to see the structure in the right form.

On Monday, September 9, 2013 12:54:46 AM UTC-4, David Pilato wrote:

My answer supposed that you are using couchdb river. Is it true?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 06:51, David Pilato <da...@pilato.fr <javascript:>> a
écrit :

It can not work out of the box.
You could think of using a script to create a new locationgeo field, and
copy lat,lon values in it:

"location" : "lat,lon"

Have a look at:
GitHub - elastic/elasticsearch-river-couchdb: CouchDB River Plugin for elasticsearch (STOPPED)

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 01:55, Oleg Aulov <aulov...@gmail.com <javascript:>> a
écrit :

Hi,

I ingest data from CouchDB into Elasticsearch and would like my lat, lon
to be mapped as geo_type, but I don't know how to do it, since my lat, lon
is not fitting any of the geo_type examples from the guide.

My data looks like this:

"location": {
"latitude": 37.632571793,
"name": "Pacifica Pier",
"longitude": -122.494277492,
"id": 5076543
}

(of course there are a bunch of other fields)

Since the "location" doesn't have an array [lat, lon] I'm not sure how to specify the type to be "geo_type".

Would something like this work?

"location":{
         "properties":{
                   "id":{"type":"long"},
                   "latitude":{"type":"geo_type"},
                   "longitude":{"type":"geo_type"},
                   "name":{"type":"string"}

          }

}

How can I combine the two values into one array?

I'd highly appreciate any suggestions or help.

Thanks!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Yeah. I guess it should work.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 07:07, Oleg Aulov aulov.oleg@gmail.com a écrit :

Hi David, thanks for a prompt reply! Yes, using CouchDB river.

Makes sense. So if there is a script in the river that moves lat, lon into an array then Elasticsearch is going to see the structure in the right form.

On Monday, September 9, 2013 12:54:46 AM UTC-4, David Pilato wrote:

My answer supposed that you are using couchdb river. Is it true?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 06:51, David Pilato da...@pilato.fr a écrit :

It can not work out of the box.
You could think of using a script to create a new locationgeo field, and copy lat,lon values in it:
"location" : "lat,lon"
Have a look at: GitHub - elastic/elasticsearch-river-couchdb: CouchDB River Plugin for elasticsearch (STOPPED)

HTH

David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 9 sept. 2013 à 01:55, Oleg Aulov aulov...@gmail.com a écrit :

Hi,

I ingest data from CouchDB into Elasticsearch and would like my lat, lon to be mapped as geo_type, but I don't know how to do it, since my lat, lon is not fitting any of the geo_type examples from the guide.

My data looks like this:

"location": {
"latitude": 37.632571793,
"name": "Pacifica Pier",
"longitude": -122.494277492,
"id": 5076543
}

(of course there are a bunch of other fields)

Since the "location" doesn't have an array [lat, lon] I'm not sure how to specify the type to be "geo_type".

Would something like this work?

"location":{
         "properties":{
                   "id":{"type":"long"},
                   "latitude":{"type":"geo_type"},
                   "longitude":{"type":"geo_type"},
                   "name":{"type":"string"}
          }
}

How can I combine the two values into one array?

I'd highly appreciate any suggestions or help.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Be careful. See

If added as a string, then it's "location" : "lat, lon"

But if added as an array, then it's "location" : [ lon, lat ] to conform
with GeoJSON.

Because GeoJSON follows the x-y-z coordinate system conventions, longitude
must come first if specified in an array of floating-point values. But
latitude comes first if the "array" is really a single string with two
comma-separated floating-point string values embedded in it.

Brian

On Monday, September 9, 2013 1:07:10 AM UTC-4, Oleg Aulov wrote:

Hi David, thanks for a prompt reply! Yes, using CouchDB river.

Makes sense. So if there is a script in the river that moves lat, lon into
an array then Elasticsearch is going to see the structure in the right form.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Thank you David and Brian.
Following your suggestions I was able to fix my index.
Here's what I did, in case someone else has a similar problem:

I installed the JavaScript plugin on my Elasticsearch cluster, restarted
the cluster and executed the following river query:

curl -XPUT 'localhost:9200/_river/test2/_meta' -d '{

"type" : "couchdb",

"couchdb" : {

   "host" : "localhost",

   "port" : 5984,

   "db" : "instagram_hurricanesandy",

   "script": "if (ctx.doc.location != undefined)

{ctx.doc.location.geopoint = [ctx.doc.location.longitude,
ctx.doc.location.latitude];}",

   "filter" : null

},

"index" : {

   "index" : "instagramsandy",

   "type" : "instagram",

   "bulk_size" : "100",

   "bulk_timeout" : "10ms"

}
}'

So, the script entry has a JavaScript code that takes Latitude, Nongtitude
String fields and creates an array of [lon, lat].
As a result, having the following index mapping worked flawlessly in
indexing geo_points:

"location":{

"properties":{

"id":{"type":"long"},

"latitude":{"type":"double"},

"longitude":{"type":"double"},

"geopoint":{"type": "geo_point"},

"name":{"type":"string"}}},

On Monday, September 9, 2013 2:55:08 PM UTC-4, InquiringMind wrote:

Be careful. See
Elasticsearch Platform — Find real-time answers at scale | Elastic

If added as a string, then it's "location" : "lat, lon"

But if added as an array, then it's "location" : [ lon, lat ] to conform
with GeoJSON.

Because GeoJSON follows the x-y-z coordinate system conventions, longitude
must come first if specified in an array of floating-point values. But
latitude comes first if the "array" is really a single string with two
comma-separated floating-point string values embedded in it.

Brian

On Monday, September 9, 2013 1:07:10 AM UTC-4, Oleg Aulov wrote:

Hi David, thanks for a prompt reply! Yes, using CouchDB river.

Makes sense. So if there is a script in the river that moves lat, lon
into an array then Elasticsearch is going to see the structure in the right
form.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.