Slow indexing geo_shape


(Georgi Ivanov) #1

Hi,
I am playing with geo_shape type .
I am experiencing very slow indexing times. For example one simple
linestring with couple of hundred points could take up to 60 seconds to
index.
I tries geohash and quadtree implementations.
With quadtree it is faster (like 50% faster) , but still not fast enough.

Using Java API (bulk indexing)

Mapping:

{
"entity": {
"properties": {
"id" : {"type": "integer"},
"track" : {"type": "geo_shape","precision":"20m", "tree":
"quadtree"},
"date" : {"type": "date"}
}
}
}

My ES cluster is tuned for indexing like follows:

index.refresh_interval: 30s
index.translog.flush_threshold_ops: 100000
indices.memory.index_buffer_size:: 15%

threadpool.bulk.queue_size: 500
threadpool.bulk.size: 100
threadpool.bulk.type: fixed

Any tips how to make indexing faster ?
My estimation is that for one day data i could index it for 10 hours (and i
need to index like 3 years of data ).

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cb6b6f20-639c-4bb1-93d9-52f81658761c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Binh Ly-2) #2

I'd experiment with the precision setting and relax it a bit to see how
much indexing speed improves.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4c031a14-95ad-411d-8511-b489a6706c57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3