hey, im try get 4500 data, but very slow 7second to complete.
im using slice scroll .
this my code
int slices = 5;
SearchSourceBuilder searchSourceBuilder = SearchSourceBuilder.searchSource();
IntStream.range(0, slices).parallel().forEach(i -> {
SliceBuilder sliceBuilder = new SliceBuilder(i, slices);
SearchResponse response = client.prepareSearch("twitter_timeline_v1").setTypes("twitter_timeline")
.setSource(searchSourceBuilder)
.setScroll(new TimeValue(60000))
.slice(sliceBuilder)
.setFrom(0)
.setSize(1000)
.setQuery(qb)
.setExplain(false)
.get();
do {
for (SearchHit responses : response.getHits()) {
List<String> tes1 = (List<String>) responses.getSource().get("tweet");
count.addAll(tes1);
}
response = client.prepareSearchScroll(response.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet();
}
while(response.getHits().getHits().length !=0);
});`
{
"state": "open",
"settings": {
"index": {
"creation_date": "1512717534505",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "85bJTl_UTsOWyMNKTLXxAw",
"version": {
"created": "5000005"
}
}
}
any solution?
thanks