Geolocation Cornfusion - Document format vs. Query/Filter format

speedopus · April 5, 2011, 8:31pm

First off, after reading the docs it seemed like any document
structured as

{
"location" : {
"lat" : 0,
"lon" : 0,
}
}

Would automatically get mapped as a type : "geo_point".

So my document should be..I tried fixed mapping and dynamic mapping
but no luck. With fixed mapping it doesn't get registered as
geo_point - clearly a failure on my end but I've no idea what.

here is an example document:
{
"rpl_edu" : {
"paused" : false,
"distanceMin": 0,
"distanceMax": 5,
"location" : {
"lat" : 33,
"lon" : -97
}
}
}

And I am trying to do a filter query like so:
{
"query" : {
"filtered" : {
"query" : {
"field" : { "paused" : "false" }
},
"filter" : {
"script" : {
"script" :"doc['location'].distance(34,-118) <
doc['distanceMax'].value"
}
}
}
}
}

If I use dynamic mapping, I get "No field found for [location]". I
have tried using rpl_edu.location, attempting to emulate
'pin.location' from the documentation but no luck. If I use fixed
mapping, it doesn't recognize it as a geo_point even though I think
i've defined it as such.

{
"rpl_edu" : {
"properties" : {
"paused" : { "type" : boolean,"index" : "analyzed" },
"distanceMin": { "type" : integer,"index" : "analyzed" },
"distanceMax": { "type" : integer,"index" : "analyzed" },
"location" : {
"type" : "geo_point",
"index" : "analyzed"
},
}
}
}

I have tried the query under "_all/_search" as well as under the index
directly "/edus/rpl_edu/_search". Tried many things. No idea why ES
won't grok my query.

I have tried the basic filter geo query defined in the doc

(which by the way there is a bug there - missing a comma after
'distance'). Query is here:

{"query": {
"filtered" : {
"query" : {
"field" : { "paused" : false }
},
"filter" : {
"geo_distance" : {
"distance" : "12km",
"location" : {
"lat" : 34,
"lon" : -118
}
}
}
}
}
}

And this is the output:
"QueryParsingException[[rpl_edus] failed to find geo_point field
[location]];"

What am I doing wrong?

kimchy · April 6, 2011, 10:11am

Geo point requires defining a geo point type. It use to be the case way back, but since a few versions ago, it is required: Elasticsearch Platform — Find real-time answers at scale | Elastic.
On Tuesday, April 5, 2011 at 11:31 PM, Christopher Smith wrote:

First off, after reading the docs it seemed like any document
structured as

{
"location" : {
"lat" : 0,
"lon" : 0,
}
}

Would automatically get mapped as a type : "geo_point".

So my document should be..I tried fixed mapping and dynamic mapping
but no luck. With fixed mapping it doesn't get registered as
geo_point - clearly a failure on my end but I've no idea what.

here is an example document:
{
"rpl_edu" : {
"paused" : false,
"distanceMin": 0,
"distanceMax": 5,
"location" : {
"lat" : 33,
"lon" : -97
}
}
}

And I am trying to do a filter query like so:
{
"query" : {
"filtered" : {
"query" : {
"field" : { "paused" : "false" }
},
"filter" : {
"script" : {
"script" :"doc['location'].distance(34,-118) <
doc['distanceMax'].value"
}
}
}
}
}

If I use dynamic mapping, I get "No field found for [location]". I
have tried using rpl_edu.location, attempting to emulate
'pin.location' from the documentation but no luck. If I use fixed
mapping, it doesn't recognize it as a geo_point even though I think
i've defined it as such.

{
"rpl_edu" : {
"properties" : {
"paused" : { "type" : boolean,"index" : "analyzed" },
"distanceMin": { "type" : integer,"index" : "analyzed" },
"distanceMax": { "type" : integer,"index" : "analyzed" },
"location" : {
"type" : "geo_point",
"index" : "analyzed"
},
}
}
}

I have tried the query under "_all/_search" as well as under the index
directly "/edus/rpl_edu/_search". Tried many things. No idea why ES
won't grok my query.

I have tried the basic filter geo query defined in the doc
Elasticsearch Platform — Find real-time answers at scale | Elastic
(which by the way there is a bug there - missing a comma after
'distance'). Query is here:

{"query": {
"filtered" : {
"query" : {
"field" : { "paused" : false }
},
"filter" : {
"geo_distance" : {
"distance" : "12km",
"location" : {
"lat" : 34,
"lon" : -118
}
}
}
}
}
}

And this is the output:
"QueryParsingException[[rpl_edus] failed to find geo_point field
[location]];"

What am I doing wrong?

speedopus · April 6, 2011, 6:33pm

Thank you for the response. As indicated I tried that as well.
Perhaps you could help me understand why that didn't work?

Here is the setup:

I place a file in /usr/local/elasticsearch/config/mappings/edus/
rpl_edu/rpl_edu.json. This is the correct location right? (I also
tried under _index in mappings.) The contents define a geo_point like
so:

{
"rpl_edu" : {
"properties" : {
"paused" : { "type" : boolean,"index" : "analyzed" },
"distanceMin": { "type" : integer,"index" : "analyzed" },
"distanceMax": { "type" : integer,"index" : "analyzed" },
"location" : {
"type" : "geo_point",
},
}
}

}

Which from the docs appears to be the correct way to define a
geo_point type (Yes, I deleted all my indexes and regenerated them
after installing the mapping).

Here is an example record written to the index that defines the above
mapping:

{
"rpl_edu" : {
"paused" : false,
"distanceMin": 0,
"distanceMax": 0,
"location" : {
"lat" : 33,
"lon" : -97
},
}

When I query against this, it cannot find the ["location"] datapoint,
even though it is clearly in the record.

E.G.
QUERY -
{"query" : {
"filtered" : {
"query" : {
"field" : { "paused" : "false" }
},
"filter" : {
"script" : {
"script" :"doc['location'].distance(33,-97) <
doc['distanceMax'].value"
}
}
}
}
}

RESULT -
"CompileException[[Error: No field found for [location]]\n[Near : {...
doc['location'].distance(33,-9 ....}]"

Here is a query that DOES work against this index:
{
"query" : {
"field" : { "location.lat" : 33 }
}
}

Which makes me think it's NOT getting picked up as a geo_point. I'm
starting to think the mapping is not working.

Lastly, I try a regular geo_distance query against this based on what
is in the docs:

RESULT:
"failed to find geo_point field [location]]"

So - clearly it is not finding the mapping. I cannot seem to figure
out why not. From the above it "seems" like I 'm doing it right. I
move my mapping definition from config/mappings/edus/rpl_edu.json to
config/mappings/_index/rpl_edu.json. I delete all my data and reseed.

RESULT:
failed to find geo_point field [location]]

I then tried defining the mapping via a json call with equivalent
results. Delete data. Reseed index.

curl -XGET 'http://localhost:9200/edus/rpl_edu/_mapping' -d
@createGeoMapping.json (the contents of the file are the same as
the mapping pasted in above)

RESULT:
"failed to find geo_point field [location]]"

I am using 0.15.1.

Am I doing something terribly wrong and mentally unsound here? I'm
usually pretty good at following instructions, and virtually never
wind up posting in forums - but even after reading the docs, following
the docs, pillaging the forums - the issue persists. Please help me
understand this. I'd really, really like this to work and I feel as
if I've tried every relevant combination I could think of, prior to
troubling the forum.

kimchy · April 6, 2011, 6:54pm

Yes, it seems like the mapping is not being picked up. You can validate the actual mapping used by using the GET mapping API.

You should place the mapping definition within the index directory, no need for a type directory as well. In your case: /usr/local/elasticsearch/config/mappings/edus/rpl_edu.json.

As a side note, not a big fan of mapping config based on files, much nicer to create them in the create index API.
On Wednesday, April 6, 2011 at 9:33 PM, Christopher Smith wrote:

Thank you for the response. As indicated I tried that as well.
Perhaps you could help me understand why that didn't work?

Here is the setup:

I place a file in /usr/local/elasticsearch/config/mappings/edus/
rpl_edu/rpl_edu.json. This is the correct location right? (I also
tried under _index in mappings.) The contents define a geo_point like
so:

{
"rpl_edu" : {
"properties" : {
"paused" : { "type" : boolean,"index" : "analyzed" },
"distanceMin": { "type" : integer,"index" : "analyzed" },
"distanceMax": { "type" : integer,"index" : "analyzed" },
"location" : {
"type" : "geo_point",
},
}
}

}

Which from the docs appears to be the correct way to define a
geo_point type (Yes, I deleted all my indexes and regenerated them
after installing the mapping).

Here is an example record written to the index that defines the above
mapping:

{
"rpl_edu" : {
"paused" : false,
"distanceMin": 0,
"distanceMax": 0,
"location" : {
"lat" : 33,
"lon" : -97
},
}

When I query against this, it cannot find the ["location"] datapoint,
even though it is clearly in the record.

E.G.
QUERY -
{"query" : {
"filtered" : {
"query" : {
"field" : { "paused" : "false" }
},
"filter" : {
"script" : {
"script" :"doc['location'].distance(33,-97) <
doc['distanceMax'].value"
}
}
}
}
}

RESULT -
"CompileException[[Error: No field found for [location]]\n[Near : {...
doc['location'].distance(33,-9 ....}]"

Here is a query that DOES work against this index:
{
"query" : {
"field" : { "location.lat" : 33 }
}
}

Which makes me think it's NOT getting picked up as a geo_point. I'm
starting to think the mapping is not working.

Lastly, I try a regular geo_distance query against this based on what
is in the docs:

RESULT:
"failed to find geo_point field [location]]"

So - clearly it is not finding the mapping. I cannot seem to figure
out why not. From the above it "seems" like I 'm doing it right. I
move my mapping definition from config/mappings/edus/rpl_edu.json to
config/mappings/_index/rpl_edu.json. I delete all my data and reseed.

RESULT:
failed to find geo_point field [location]]

I then tried defining the mapping via a json call with equivalent
results. Delete data. Reseed index.

curl -XGET 'http://localhost:9200/edus/rpl_edu/_mapping' -d
@createGeoMapping.json (the contents of the file are the same as
the mapping pasted in above)

RESULT:
"failed to find geo_point field [location]]"

I am using 0.15.1.

Am I doing something terribly wrong and mentally unsound here? I'm
usually pretty good at following instructions, and virtually never
wind up posting in forums - but even after reading the docs, following
the docs, pillaging the forums - the issue persists. Please help me
understand this. I'd really, really like this to work and I feel as
if I've tried every relevant combination I could think of, prior to
troubling the forum.