I m planning to create a graph to visualize the relation between different cabs pickups location in Nyc city.
so i choosed the index named "green" (for green cabs) and the field pickup_location (which is type is geohash), the result is 500 server error !?
Can you share the mapping and the stacktrace?
I expect you want to map the value as a keyword type if you are using it as a location identifier - geo fields are optimised for spatial range queries rather than ID-type lookups.
this is the stacktrace i took a screenshot
what you mean by mapping ? template mapping for geopoint ?
Mappings are effectively the schema for your index - see Mapping | Elasticsearch Guide [8.11] | Elastic
Here's an example of trips from JFK to Brooklyn and Queens which you can paste into Kibana's console:
DELETE test
PUT test
{
"mappings": {
"trip": {
"properties": {
"geo": {
"type": "geo_point",
"fields":{
"keyword":{
"type":"keyword"
}
}
}
}
}
}
}
POST test/trip/_bulk
{"index":{}}
{ "geo":["dr5x2bn", "dr5x8v1"] }
{"index":{}}
{ "geo":["dr5x2bn", "dr5rmks"]}
GET test/_search
{
"size": 0,
"aggs": {
"terms": {
"terms": {
"field": "geo.keyword"
}
}
}
}
Note that I mapped the JSON value as both a geo_point field and a keyword field - one is for spatial queries, the other for ID-based queries of the sort Graph uses.
Ok, understood i this is the mapping of the index green
{
"green": {
"mappings": {
"traj": {
"properties": {
"@timestamp": {
"type": "date"
},
"@version": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Dropoff_location": {
"type": "geo_point"
},
"Extra": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Fare_amount": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Lpep_dropoff_datetime": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"MTA_tax": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Passenger_count": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Payment_type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"PickupDate": {
"type": "date"
},
"Pickup_location": {
"type": "geo_point"
},
"RateCodeID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Store_and_fwd_flag": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Tip_amount": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Tolls_amount": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Total_amount": {
"type": "float"
},
"Trip_distance": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Trip_type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"VendorID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"host": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lpep_pickup_datetime": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"path": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"_default_": {
"properties": {
"Dropoff_location": {
"type": "geo_point"
},
"Pickup_location": {
"type": "geo_point"
}
}
}
}
}
}so ?
I suggest you create a new field called "my_location_keywords" mapped as a keyword type.
You should pass docs, as in my example, where the pickup and dropoff points are expressed as 2 truncated geohashes in the "my_location_keywords" array. Why truncated? Full-resolution geohashes will likely lead to a graph with millions of nodes each containing exactly one doc per connection - every full-precision pickup/dropoff location may be unique. If you truncate the geohash according to this table you'll "round up" to a higher level view of the data.
If your original data is lat/lon you can find geohash implementations for a variety of languages.
Ok, I did as you told me:
i changed my mapping for the index like this :
PUT _template/test
{
"order":0,
"index_patterns":"test",
"mappings":{
"_default_":{
"properties":{
"Pickup_location":{
"type":"geo_point"
,"fields":{
"keyword":{
"type":"keyword"
}
}
},
"Dropoff_location":{
"type":"geo_point"
,"fields":{
"keyword":{
"type":"keyword"
}
}
}
}
}
}
}
then i indexed with logstash the test index.
the result i still don't see the graph created when i choice test as index and pickup_location.keyword as a field, but the error is gone.
i don't undesrstand the truncated part of geohash since the ingestion is done by logstash
You need to write some code or dig deeper into what logstash can do for you in terms of document preparation.
ok , but even your example dosen't work for me.
i chosed the pickup_location as a field
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.