# How to aggregate many events?

**URL:** https://discuss.elastic.co/t/how-to-aggregate-many-events/203836
**Category:** Logstash
**Created:** [October 16, 2019, 12:38pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836 "2019-10-16T12:38:06Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![RickT](https://avatars.discourse-cdn.com/v4/letter/r/3bc359/32.png) [@RickT](https://discuss.elastic.co/u/RickT)
#### Post date: [October 16, 2019, 12:38pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/1 "2019-10-16T12:38:06Z")

</div>

hi,  
i'm searching how to aggregate 2 or more events issued from a **log file**. The events have an common id. This is the criteria to aggregate events.

I has tried the example of logstash 6.8 documentation, but it doesn't work ! ☹

![image](https://us1.discourse-cdn.com/elastic/original/3X/a/1/a1edf5dc9cd84255794153c45a180fc05ee6fef3.png)

And I would like these 2 result events to push them into elasticsearch :

{ "country\_name": "France", "towns": [{"town\_name": "Paris"}, {"town\_name": "Marseille"}] }  
{ "country\_name": "USA", "towns": [{"town\_name": "New-York"}] }  
But in fact, the result obtained is 3 lines in Elastic index ! Why ?

i'm using this filter :  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/0/00054c3a28aed5ea91a05665bc810dd5dbe4f3f6.png)  
info : logstash 6.8.3, Elastic 6.8

is there someone to help me please ?  
Thanks

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 16, 2019, 1:09pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/2 "2019-10-16T13:09:18Z")

</div>

Have you disabled pipeline.java\_execution?

---

<div class="post-metadata">

### Author: ![RickT](https://avatars.discourse-cdn.com/v4/letter/r/3bc359/32.png) [@RickT](https://discuss.elastic.co/u/RickT)
#### Post date: [October 16, 2019, 2:47pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/3 "2019-10-16T14:47:50Z")

</div>

Hi Badger,

No **pipeline.java** , but **logstash.pipeline** declared :

[2019-10-16T15:59:01,392][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified  
[2019-10-16T15:59:01,410][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.8.3"}  
[2019-10-16T15:59:12,385][INFO][**logstash.pipeline**] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>4, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50}

is there a relation with the malfunction ?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 16, 2019, 3:01pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/4 "2019-10-16T15:01:54Z")

</div>

Add

```
pipeline.java_execution: false

```

in logstash.yml or add '--java-execution false' to the command line to work around [this](https://github.com/elastic/logstash/issues/10938) issue. Also you must set pipeline.workers to 1.

---

<div class="post-metadata">

### Author: ![RickT](https://avatars.discourse-cdn.com/v4/letter/r/3bc359/32.png) [@RickT](https://discuss.elastic.co/u/RickT)
#### Post date: [October 16, 2019, 3:43pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/5 "2019-10-16T15:43:50Z")

</div>

i tried :

**logstash -f D:\LogstashConf\fic\_info2.conf --java-execution false --pipeline.workers 1**

[2019-10-16T17:16:28,385][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"6.8.3"}  
[2019-10-16T17:16:39,973][INFO][logstash.pipeline] Starting pipeline {:pipeline\_id=\>"main", " **pipeline.workers"=\>1,**"pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>50}

But the result is always 3 events in Elastic. Too bad ! 🙁

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 13, 2019, 3:44pm UTC](https://discuss.elastic.co/t/how-to-aggregate-many-events/203836/6 "2019-11-13T15:44:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
