# Elasticsearch correlation getting failed

**URL:** <https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927>\
**Category:** Logstash\
**Created:** [September 30, 2016, 2:05pm UTC](https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927 "2016-09-30T14:05:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rajkamalkool6](https://avatars.discourse-cdn.com/v4/letter/r/e9c0ed/32.png) [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\
**Post date:** [September 30, 2016, 2:05pm UTC](https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927/1 "2016-09-30T14:05:00Z")

</div>

Hi,

I have request and response files which I need to process and correlate both the file fields and to correlate it. In my scenario filebeat picking up the files in random hence sometime my response files getting read instead of request files. So to cover all the scenarios I have the logic both in my response and request logstash configuration files. Below are my sample code.

**Request section:**

```
filter{
.......

elasticsearch {
			  hosts => ["xxxxx.xx:9200"]
			  query => "fileidres:%{fileidreq}"
			  fields => [
			        "status","status",
				"creationTime","creationTime",
				"fileidres","fileidres"
			       ]
}

if [status] == "CODE"
			{ <Some logic>
}
}

```

`fileidres` is the field created in `response` block and` fileidreq` was created in` request` block

**Response Section:**

> ```
> filter{
> grok {
> match => { "responseDesc" => "(?<status>CODE)" }  
> }
> }
> 
> ```

1. I would like to know is thee any option to make` Filebeat` or `Logstash` to process the `request files` first before` response file`s.

2. Is there any attribute available in `Elasticsearc`h filter to collect all data if the query is success.

Thanks in advance

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 5, 2016, 6:04am UTC](https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927/2 "2016-10-05T06:04:53Z")

</div>

Sorry, but this approach to merging events is fundamentally broken. You need to be able to synchronize so that the entries in one file is read before the other file's entries and Logstash has no mechanisms for that.

---

<div class="post-metadata">

**Author:** ![rajkamalkool6](https://avatars.discourse-cdn.com/v4/letter/r/e9c0ed/32.png) [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\
**Post date:** [October 6, 2016, 6:02am UTC](https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927/4 "2016-10-06T06:02:49Z")

</div>

How can I synchronize the events?. Is my elasticsearch aggregation is correct?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:35am UTC](https://discuss.elastic.co/t/elasticsearch-correlation-getting-failed/61927/5 "2017-07-06T04:35:30Z")

</div>