# Multiple docker container for multiple pipeline of logstash **URL:** \ **Category:** Logstash\ **Created:** [August 19, 2020, 10:18pm UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684 "2020-08-19T22:18:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [August 19, 2020, 10:18pm UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/1 "2020-08-19T22:18:45Z")
Currently I run 20 pipeline in a system some just pulls data few times a day some just once a day some every few minute I was thinking to run multiple docker image on a single system, where slower pipeline is running on one. and very active one running by self. system has 24 core of cpu in a system and 96gig ram. is it possible by assigning each image a X cpu and Y memory? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 19, 2020, 10:56pm UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/2 "2020-08-19T22:56:23Z")
You can adjust things like heap space for Logstash, but CPU is a little harder. You may want to look at what docker can provide on that aspect. ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [August 20, 2020, 2:10am UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/3 "2020-08-20T02:10:37Z")
busiest pipeline ingest about 3million record in 24 hour. 1.2gig data only should one cpu or one core per pipeline work? I do have multiple system where I can do this. all smaller pipleline I can put it togather with 2-4 cpu and it should do the job. ---
**Author:** ![cknz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cknz/32/9640_2.png) [@cknz](https://discuss.elastic.co/u/cknz)\ **Post date:** [August 20, 2020, 4:29am UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/4 "2020-08-20T04:29:13Z")
When you start thinking of multiple containers, you start thinking about orchestration. What are you using to start the containers? Docker does have its own resource usage constraint system, which uses the Linux Control Groups subsystem. > **[Runtime options with Memory, CPUs, and GPUs](https://docs.docker.com/config/containers/resource_constraints/)** > > By default, a container has no resource constraints and can use as much of a given resource as the host’s kernel scheduler allows. Docker provides ways to control how much... If you want to look at monitoring; if you're using Prometheus you could look at the likes of cAdvisor ([https://github.com/google/cadvisor](https://github.com/google/cadvisor)) ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [August 20, 2020, 12:56pm UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/5 "2020-08-20T12:56:32Z")
I am in testing phase at this time used docker build to create my image via dockerfile provided by ELK using docker run to start it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 17, 2020, 12:56pm UTC](https://discuss.elastic.co/t/multiple-docker-container-for-multiple-pipeline-of-logstash/245684/6 "2020-09-17T12:56:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.