Hi Team,
I want to fetch list of agent ephemeral Id's integrated for a particular agent along with the client ip and process.pid. For the same, I have implemented the EQL using aggregations which gives me the output as expected.
But in case of EQL's java implementation , It gives me entire list of emphermal ID irrespective of it being within the timestamp range.
Adding the snippet of both EQL and its java implementation
EQL
GET traces-apm*,metrics-apm*/_search
{
"size": 0,
"fields": [
"service.name",
"@timestamp"
],
"query": {
"bool": {
"filter": [
{ "range": {
"@timestamp": {
"gte": "now-1h",
"lte": "now"
}
}},{
"bool":{
"must":[
{
"term":{
"service.name" : "my_service_name"
}
}
]
}
}
]
}
},
"_source": false,
"aggs": {
"service_name": {
"terms": {
"field": "service.name",
"size": 100
},
"aggs": {
"agent_ephemeral_id": {
"terms": {
"field": "agent.ephemeral_id"
}, "aggs":{
"client_ip" :{
"terms" :{
"field" :"client.ip"
}
},"process_pid" : {
"terms" :{
"field" : "process.pid"
}
}
}
}
}
}
}
}
Java Implementation of above EQL query.
List<String> eqlQueryagentEphermalIdList=new ArrayList<>();
try (RestHighLevelClient client = new RestHighLevelClient(builder)) {
SearchRequest searchRequest = new SearchRequest("traces-apm*,metrics-apm*");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().fetchSource(false)
.fetchField("service.name").fetchField("@timestamp");
searchSourceBuilder
.query(QueryBuilders.boolQuery()
.filter(QueryBuilders.rangeQuery("@timestamp").gte("now-1h").lt("now")))
.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("service.name", agentName)))
.aggregation(AggregationBuilders.terms("service_name").field("service.name")
.subAggregation(AggregationBuilders.terms("agent_ephemeral_id").field("agent.ephemeral_id").size(1000)
.subAggregation(AggregationBuilders.terms("client_ip").field("client.ip").size(1000)))
.subAggregation(AggregationBuilders.terms("process_pid").field("process.pid").size(1000)));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
Terms terms = searchResponse.getAggregations().get("service_name");
for(Terms.Bucket bucket : terms.getBuckets()) {
System.out.println("bucket======"+bucket.getAggregations().toString());
Terms sub_terms=bucket.getAggregations().get("agent_ephemeral_id");
for(Terms.Bucket termsBucket : sub_terms.getBuckets()) {
System.out.println("bucket Aggregation======"+termsBucket.getKeyAsString() );
eqlQueryagentEphermalIdList.add(termsBucket.getKeyAsString());
}
} List<String> eqlQueryagentEphermalIdList=new ArrayList<>();
try (RestHighLevelClient client = new RestHighLevelClient(builder)) {
SearchRequest searchRequest = new SearchRequest("traces-apm*,metrics-apm*");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().fetchSource(false)
.fetchField("service.name").fetchField("@timestamp");
searchSourceBuilder
.query(QueryBuilders.boolQuery()
.filter(QueryBuilders.rangeQuery("@timestamp").gte("now-1h").lt("now")))
.query(QueryBuilders.boolQuery().must(QueryBuilders.termQuery("service.name", agentName)))
.aggregation(AggregationBuilders.terms("service_name").field("service.name")
.subAggregation(AggregationBuilders.terms("agent_ephemeral_id").field("agent.ephemeral_id").size(1000)
.subAggregation(AggregationBuilders.terms("client_ip").field("client.ip").size(1000)))
.subAggregation(AggregationBuilders.terms("process_pid").field("process.pid").size(1000)));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
Terms terms = searchResponse.getAggregations().get("service_name");
for(Terms.Bucket bucket : terms.getBuckets()) {
System.out.println("bucket======"+bucket.getAggregations().toString());
Terms sub_terms=bucket.getAggregations().get("agent_ephemeral_id");
for(Terms.Bucket termsBucket : sub_terms.getBuckets()) {
System.out.println("bucket Aggregation======"+termsBucket.getKeyAsString() );
eqlQueryagentEphermalIdList.add(termsBucket.getKeyAsString());
}
}
Can someone please advise in what has to be corrected in the java implementation of the EQL.
I tried providing size to the sub aggregations but that did not work.
Thanks & Regards,
Namita