Help: Beginners Guide needed


(Thorsten) #1

Hi there,
I'm looking for a dead simple, step by step explanation of how to get
started with elaticsearch.

What I've done so far:

I went through the examples on the github readme, and created the twitter
items.

curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" : "Shay
Banon" }'
curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'
curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'

What I want to do:

Add an extra field "location" to the tweets, and make it a geo_point.

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?",
"location" : {
"lat" : 41.12,
"lon" : 8.21
}
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?" ,
"location" : {
"lat" : 50.12,
"lon" : 8.34
}
}'

I tried adding a mapping, but it does not seem to work.

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
'

When I try to filter the results, still all tweets are returned, no matter
how close I set the distance in the filter.

{
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "2km",
"tweet.location" : {
"lat" : 30.00,
"lon" : 50.00
}
}
}
}
}

What am I doing wrong? Thanks for your help

Thorsten


(Ivan Brusic) #2

You cannot change a field's type after a document with that field has
been indexed. Try explicitly creating the index first with that
mapping, then add your documents.

--
Ivan

On Wed, May 16, 2012 at 9:53 AM, Thorsten thorsten1983@googlemail.com wrote:

Hi there,
I'm looking for a dead simple, step by step explanation of how to get
started with elaticsearch.

What I've done so far:

I went through the examples on the github readme, and created the twitter
items.

curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" : "Shay
Banon" }'
curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'
curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'

What I want to do:

Add an extra field "location" to the tweets, and make it a geo_point.

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?",
"location" : {
"lat" : 41.12,
"lon" : 8.21
}
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?" ,
"location" : {
"lat" : 50.12,
"lon" : 8.34
}
}'

I tried adding a mapping, but it does not seem to work.

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
'

When I try to filter the results, still all tweets are returned, no matter
how close I set the distance in the filter.

{
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "2km",
"tweet.location" : {
"lat" : 30.00,
"lon" : 50.00
}
}
}
}
}

What am I doing wrong? Thanks for your help

Thorsten


(Thorsten) #3

I'm sorry, but it seems I need it more fool-prove :wink:

I reseted elasticsearch, and tried to follow the docshttp://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html but
I'm a bit confused with how to edit the code given there:

curl -XPOST localhost:9200/test -d '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"field1" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}'

I removed the "settings" and "_source":

curl -XPOST localhost:9200/twitter/tweet -d '{
"mappings" : {
"type1" : {
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
}
}'

And entered the tweets again, but no luck with geo filtering.

Do I need to change "type1" as well? And if so, into what?

I might appear stupid, but the docs aren't quite clear to me... :frowning:

Thanks

Am Mittwoch, 16. Mai 2012 18:57:15 UTC+2 schrieb Ivan Brusic:

You cannot change a field's type after a document with that field has
been indexed. Try explicitly creating the index first with that
mapping, then add your documents.

--
Ivan

On Wed, May 16, 2012 at 9:53 AM, Thorsten wrote:

Hi there,
I'm looking for a dead simple, step by step explanation of how to get
started with elaticsearch.

What I've done so far:

I went through the examples on the github readme, and created the
twitter
items.

curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" :
"Shay
Banon" }'
curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'
curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'

What I want to do:

Add an extra field "location" to the tweets, and make it a geo_point.

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?",
"location" : {
"lat" : 41.12,
"lon" : 8.21
}
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?" ,
"location" : {
"lat" : 50.12,
"lon" : 8.34
}
}'

I tried adding a mapping, but it does not seem to work.

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
'

When I try to filter the results, still all tweets are returned, no
matter
how close I set the distance in the filter.

{
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "2km",
"tweet.location" : {
"lat" : 30.00,
"lon" : 50.00
}
}
}
}
}

What am I doing wrong? Thanks for your help

Thorsten


(Ivan Brusic) #4

In your case, type1 should be tweet.

On Wed, May 16, 2012 at 10:25 AM, Thorsten thorsten1983@googlemail.com wrote:

I'm sorry, but it seems I need it more fool-prove :wink:

I reseted elasticsearch, and tried to follow the docs but I'm a bit confused
with how to edit the code given there:

curl -XPOST localhost:9200/test -d '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"type1" : {
"_source" : { "enabled" : false },
"properties" : {
"field1" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}'

I removed the "settings" and "_source":

curl -XPOST localhost:9200/twitter/tweet -d '{
"mappings" : {
"type1" : {
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
}
}'

And entered the tweets again, but no luck with geo filtering.

Do I need to change "type1" as well? And if so, into what?

I might appear stupid, but the docs aren't quite clear to me... :frowning:

Thanks

Am Mittwoch, 16. Mai 2012 18:57:15 UTC+2 schrieb Ivan Brusic:

You cannot change a field's type after a document with that field has
been indexed. Try explicitly creating the index first with that
mapping, then add your documents.

--
Ivan

On Wed, May 16, 2012 at 9:53 AM, Thorsten wrote:

Hi there,
I'm looking for a dead simple, step by step explanation of how to get
started with elaticsearch.

What I've done so far:

I went through the examples on the github readme, and created the
twitter
items.

curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" :
"Shay
Banon" }'
curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'
curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'

What I want to do:

Add an extra field "location" to the tweets, and make it a geo_point.

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?",
"location" : {
"lat" : 41.12,
"lon" : 8.21
}
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?" ,
"location" : {
"lat" : 50.12,
"lon" : 8.34
}
}'

I tried adding a mapping, but it does not seem to work.

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"properties" : {
"location" : {
"type" : "geo_point"
}
}
}
'

When I try to filter the results, still all tweets are returned, no
matter
how close I set the distance in the filter.

{
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "2km",
"tweet.location" : {
"lat" : 30.00,
"lon" : 50.00
}
}
}
}
}

What am I doing wrong? Thanks for your help

Thorsten


(Thorsten) #5

I've done that as well now, but no success.

Looking at the mappings, the mapping is not listed as "geo".

curl -XGET 'http://localhost:9200/twitter/tweet/_mapping?pretty=true'
{
"tweet" : {
"properties" : {
"location" : {
"dynamic" : "true",
"properties" : {
"lat" : {
"type" : "double"
},
"lon" : {
"type" : "double"
}
}
},
"mappings" : {
"dynamic" : "true",
"properties" : {
"tweet" : {
"dynamic" : "true",
"properties" : {
"properties" : {
"dynamic" : "true",
"properties" : {
"location" : {
"dynamic" : "true",
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
}
}
}
}
},
"message" : {
"type" : "string"
},
"postDate" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"user" : {
"type" : "string"
}
}
}
}

BTW: How can I get rid of my email appearing in these posts? Thats not
quite helpful from google for preventing spam...


(Ivan Brusic) #6

Not quite there. :slight_smile:

Your mapping is way off. It looks like your post a mapping as a
document. The "property property" is a good indication:

"dynamic" : "true",
"properties" : {
"properties" : {
"dynamic" : "true",

Your put mapping is missing _mapping
curl -XPOST localhost:9200/twitter/tweet -d '{
"mappings" : {

I do not use the REST API for mappings, but it should look like this
example: http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html

--
Ivan

On Wed, May 16, 2012 at 10:55 AM, Thorsten thorsten1983@googlemail.com wrote:

I've done that as well now, but no success.

Looking at the mappings, the mapping is not listed as "geo".

curl -XGET 'http://localhost:9200/twitter/tweet/_mapping?pretty=true'
{
"tweet" : {
"properties" : {
"location" : {
"dynamic" : "true",
"properties" : {
"lat" : {
"type" : "double"
},
"lon" : {
"type" : "double"
}
}
},
"mappings" : {
"dynamic" : "true",
"properties" : {
"tweet" : {
"dynamic" : "true",
"properties" : {
"properties" : {
"dynamic" : "true",
"properties" : {
"location" : {
"dynamic" : "true",
"properties" : {
"type" : {
"type" : "string"
}
}
}
}
}
}
}
}
},
"message" : {
"type" : "string"
},
"postDate" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"user" : {
"type" : "string"
}
}
}
}

BTW: How can I get rid of my email appearing in these posts? Thats not quite
helpful from google for preventing spam...


(Thorsten) #7

Thanks a lot for your patience, got it working now :smiley:

For all others who are interested, here's the summary:

HowTo: Elasticsearch with GeoData:

1. Create empty index "twitter"

curl -XPUT 'http://localhost:9200/twitter/'

2. create geo_point mapping for the actual "tweet"

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"tweet" : {
"properties" : {
"location" : {"type" : "geo_point"}
}
}
}'

3. Let's check if the mapping was set

curl -XGET 'http://localhost:9200/twitter/tweet/_mapping?pretty=true'

(should return ..."location: type: geo_point"...)

4. Post some tweets, with geo data

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?",
"location" : {
"lat" : 41.12,
"lon" : 8.21
}
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?" ,
"location" : {
"lat" : 50.12,
"lon" : 8.34
}
}'

5.a) Filter by distance

curl -XGET 'http://localhost:9200/twitter/tweet/_search' -d '{
"query": {
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "200km",
"tweet.location" : {
"lat" : 40.00,
"lon" : 8.00
}
}
}
}
}
}'

5.b) sort by distance, and get distance value back in results

curl -XGET 'http://localhost:9200/twitter/tweet/_search' -d '{
"sort" : [
{
"_geo_distance" : {
"tweet.location" : {
"lat" : 41,
"lon" : 8
},
"order" : "asc",
"unit" : "km"
}
}
],
"query" : {
"match_all" : {}
}
}'


(system) #8