We are doing following query which is a range query and we apply sort by
timestamp criteria:
EndUserTxnReportSearchCriteria criteria = new
EndUserTxnReportSearchCriteria();
criteria.setJurhash(getReqAccount().getJurHash());
for(String operation: operationList)
criteria.addOperation(operation);
criteria.setSortDir("DESC");
criteria.setSortBy("Date");
Date endDate = Calendar.getInstance(getUserTimeZone()).getTime();
criteria.setEndDate(endDate);
Calendar startDateCal = Calendar.getInstance();
startDateCal.add(Calendar.YEAR, -1);
Date startDate = startDateCal.getTime();
criteria.setStartDate(startDate);
QueryBuilder jh =
QueryBuilders.termQuery(ElasticSearchTransactionTypeUtil.Fields.account.toString(),
criteria.getJurhash());
BoolQueryBuilder boolQb = QueryBuilders.boolQuery().must(jh);
// Add operation if present
if (criteria.getOperations()!=null &&
!criteria.getOperations().isEmpty()){
for(String operation: criteria.getOperations())
boolQb.should(QueryBuilders.termQuery(ElasticSearchTransactionTypeUtil.Fields.operation.toString(),operation));
boolQb.minimumNumberShouldMatch(1);
}
// Add userID to query
if (criteria.getExtUserId()!=null &&
!criteria.getExtUserId().equals("")){
//TODO: For demo, comment out wildcard search
//QueryBuilder uid =
QueryBuilders.wildcardQuery(ElasticSearchTransactionTypeUtil.Fields.user.toString(),
"" + criteria.getExtUserId().toLowerCase() + "");
QueryBuilder uid =
QueryBuilders.wildcardQuery(ElasticSearchTransactionTypeUtil.Fields.user.toString(),
"" + criteria.getExtUserId().toLowerCase() + "");
boolQb = boolQb.must(uid);
}
// Build query
QueryBuilder qb = QueryBuilders.filteredQuery(boolQb,
FilterBuilders.rangeFilter(ElasticSearchTransactionTypeUtil.Fields.timeStamp.toString())
.from(criteria.getStartDate())
.to(criteria.getEndDate())
.includeLower(true)
.includeUpper(false));
// Get client
TransportClient client = getClient();
if(client != null){
try{
if(log.isDebugEnabled())
log.debug("Query:" + new
String(qb.buildAsBytes(XContentType.JSON)));
// Get response
SearchRequestBuilder requestBuilder= client.prepareSearch()
.setOperationThreading(SearchOperationThreading.THREAD_PER_SHARD)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(qb.buildAsBytes())
.setFrom(firstResult - 1)
.setSize(pageSize)
.setIndices(esIndices)
*.addSort(getSortBy(criteria.getSortBy()),
getSortDir(criteria.getSortDir())) *
.setExplain(false);
SearchResponse response =
requestBuilder.execute().actionGet(getTimeoutinmillis());
if(log.isDebugEnabled())
log.debug("SearchResponse:" + response.toString());
List<EventLog> eventLogList = new ArrayList<EventLog>();
try {
SearchHit[] hits = (response==null || response.getHits()==
null)?null:response.getHits().getHits();
if(hits!=null && hits.length > 0) {
EventLog eventLog;
for(int i=0;i<hits.length;i++){
Map<String,Object> map =
hits[i].sourceAsMap();
eventLog =
ElasticSearchTransactionTypeUtil.convertFromESDataToEventLog(map);
if(log.isDebugEnabled())
log.debug("Adding event log " + eventLog);
eventLogList.add(eventLog);
}
return eventLogList;
}
} catch (Exception e) {
log.error("Error while parsing the results from elastic
search.");
throw new RuntimeException(e);
}
return eventLogList;
}
finally{
//The client should never close because of the client is
singleton.
//if(client!=null) client.close();
}
}
return null;
}
Timestamp is a very high cardinality field (Almost unique and probably we
have a lots and lots of such unique terms in our data). I think sorting by
timestamp is something that is causing problems with these queries..
When we search for last 2 years worth of data and do a search, the search
just fails to give us back results within 20s timeout. We face this issue
intermittantly and we are trying to debug why is this happening.
Currently we use Elastic search 0.18.3 .
Is there a better way of writing above query to get data for reporting?
Is there some functionality like limite() in Elastic Search Search API?
Looks like we also need to move to better version of lucene to improve on
memory usage (lucene 3.6+)
Thanks and regards,
Girish Khadke
On Thursday, February 14, 2013 3:20:37 AM UTC-8, Clinton Gormley wrote:
- How do we upgrade a running production cluster without having a
downtime (rolling upgrade)?
I wrote up an explanation of how we did a major upgrade without downtime
here:
Upgrading a running elasticsearch cluster · GitHub
clint
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.