[hadoop] Store GeoJson using PIG

Elastic Stack Elasticsearch
1

How can I store GeoJson using PIG?

I tried to use TOBAG(lon, lat), however I got in the location stored in the
following form:

location: [
{
0: 23.80323889
}
{
0: 44.31903611
}
]

Regards,
Dumitru

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/56fa268b-d3be-499f-a93d-b49e74779208%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Anyone tried this before?

Regards,
Dumitru

On Monday, February 3, 2014 9:49:10 PM UTC+2, Dumitru Pascu wrote:

How can I store GeoJson using PIG?

I tried to use TOBAG(lon, lat), however I got in the location stored in
the following form:

location: [
{
0: 23.80323889
}
{
0: 44.31903611
}
]

Regards,
Dumitru

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0d4c090b-8ae3-4a00-bf11-8cbf8a4b5da0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Can you post your Pig script and your index mapping? Note that for non-native types (Pig doesn't support IP or geo) you
should define the mapping in advanced in ES since es-hadoop will map it as strings as otherwise.

On 2/5/2014 8:30 PM, Dumitru Pascu wrote:

Anyone tried this before?

Regards,
Dumitru

On Monday, February 3, 2014 9:49:10 PM UTC+2, Dumitru Pascu wrote:

How can I store GeoJson using PIG?

I tried to use TOBAG(lon, lat), however I got in the location stored in the following form:

location: [
{
        0: 23.80323889
}
{
         0: 44.31903611
}
]

Regards,
Dumitru

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0d4c090b-8ae3-4a00-bf11-8cbf8a4b5da0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/530E65BF.8040503%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Index mapping (defined before running the PIG script):

curl -XPOST localhost:9200/location -d '{
"mappings" : {
"cell" : {
"properties" : {
"mapLocation" : {"type" : "geo_point" },
"name" : {"type" : "string"}
}
}
}
}'

PIG script:

location = load '/user/root/locations/cells_location.txt' USING
PigStorage('|') as (name: chararray, latitude: double, longitude: double);
locationToStore = FOREACH location GENERATE name, TOBAG(longitude,
latitude) as mapLocation;
STORE locationToStore INTO 'location/cell' USING
org.elasticsearch.hadoop.pig.EsStorage();

For the last test I used elasticsearch-hadoop-1.3.0.M2.jar

Thanks!
Dumitru

On Thu, Feb 27, 2014 at 12:07 AM, Costin Leau costin.leau@gmail.com wrote:

Can you post your Pig script and your index mapping? Note that for
non-native types (Pig doesn't support IP or geo) you should define the
mapping in advanced in ES since es-hadoop will map it as strings as
otherwise.

On 2/5/2014 8:30 PM, Dumitru Pascu wrote:

Anyone tried this before?

Regards,
Dumitru

On Monday, February 3, 2014 9:49:10 PM UTC+2, Dumitru Pascu wrote:

How can I store GeoJson using PIG?

I tried to use TOBAG(lon, lat), however I got in the location stored

in the following form:

location: [
{
        0: 23.80323889
}
{
         0: 44.31903611
}
]

Regards,
Dumitru

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to

elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0d4c090b-
8ae3-4a00-bf11-8cbf8a4b5da0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/3XWKWd-XiPk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/530E65BF.8040503%40gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPUuK1YHLXOh7iP6NiCJQv64m3kva3bitr1S3OTmB1htLpgaUQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.