# Possible ELK merges

**URL:** <https://discuss.elastic.co/t/possible-elk-merges/296715>\
**Category:** Elasticsearch\
**Created:** [February 9, 2022, 11:25am UTC](https://discuss.elastic.co/t/possible-elk-merges/296715 "2022-02-09T11:25:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![marc0069urca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc0069urca/32/100770_2.png) [@marc0069urca](https://discuss.elastic.co/u/marc0069urca)\
**Post date:** [February 9, 2022, 11:25am UTC](https://discuss.elastic.co/t/possible-elk-merges/296715/1 "2022-02-09T11:25:07Z")

</div>

Hello community,  
I wanted to know which possibility was better for SQL data merging in ELK?

Also I find myself with hundred of resource to aggregate, is it better to do aggregates or jbc streaming between 2 data entries or to do it at the end of processing all entries?  
First option :

```auto
- pipeline.id: "2-pipeline-moodle"
  path.config: "/etc/logstash/conf.d/pipeline/test/2-pipeline-moodle/{11-utilisateur-input,12-utilisateur-cours-input,13-cours-nom-input,14-cours-nom-achevement-input,15-cours-critere-input,16-cours-critere-achevement-input,17-moodle-input-calendrier,18-moodle-input-notification-resultat,19-moodle-input-cours-module,20-moodle-input-cours-module-achevement,21-moodle-input-trax-module,24-filter,25-output}.conf"
  pipeline.workers: 1

```

Second option:

```auto
- pipeline.id: "3-pipeline-moodle"
  path.config: "/etc/logstash/conf.d/pipeline/test/3-pipeline-moodle/{11-utilisateur-input,12-utilisateur-cours-input,121-filter,13-cours-nom-input,14-cours-nom-achevement-input,15-cours-critere-input,151-filter,16-cours-critere-achevement-input,17-moodle-input-calendrier,18-moodle-input-notification-resultat,181-filter,19-moodle-input-cours-module,20-moodle-input-cours-module-achevement,21-moodle-input-trax-module,212-filter,25-output}.conf"
  pipeline.workers: 1

```

Last question, as I merge a set of 100 tables given in ELK, I arrive at extreme cases of very consequent hits more than 1 million when I do not aggregate or millions of data in the available fields when I aggregate, as this dashboard service is intended for neophyte public of the digital, it would be perhaps more interesting to concentrate the data necessary to an index for a specific request on the data?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 9, 2022, 11:25am UTC](https://discuss.elastic.co/t/possible-elk-merges/296715/2 "2022-03-09T11:25:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.