# Limit to number of pipelines **URL:** \ **Category:** Logstash\ **Created:** [May 24, 2018, 8:20pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221 "2018-05-24T20:20:00Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![dalamar666](https://avatars.discourse-cdn.com/v4/letter/d/e9bcb4/32.png) [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Post date:** [May 24, 2018, 8:20pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221/1 "2018-05-24T20:20:00Z")
Hi, I have 5 pipelines on my machine. I have 8 CPU's and enough RAM and disk space for them. Only 3 of the pipelines are reporting data and creating indices. I have verified all of my configurations to make sure I did not miss anything and I have checked with tshark to make sure that I was getting data. Logstash 6.2 Unbuntu Server 16.04. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 26, 2018, 5:50am UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221/2 "2018-05-26T05:50:51Z")
What do the pipelines look like? Do they have any identical inputs that could cause problems? Is there anything in the Logstash logs? ---
**Author:** ![dalamar666](https://avatars.discourse-cdn.com/v4/letter/d/e9bcb4/32.png) [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Post date:** [May 29, 2018, 3:46pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221/3 "2018-05-29T15:46:47Z")
They all have identical inputs. The inputs are for netflow and they are all using different ports to keep things separate. The logs show things appearing to be working as normal however, my indices are not increasing with data. I do see this error ``` [2018-05-29T11:34:39,815][ERROR][logstash.pipeline] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {:pipeline_id=>"edpnc", "exception"=>"Java heap space", "backtrace"=>["java.util.Arrays.copyOfRange(Arrays.java:3664)", "java.lang.String.(String.java:207)", "java.lang.StringBuilder.toString(StringBuilder.java:407)", "org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:247)", "org.yaml.snakeyaml.reader.StreamReader.peek(StreamReader.java:160)", "org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalarNonSpaces(ScannerImpl.java:1865)", "org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalar(ScannerImpl.java:1845)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchFlowScalar(ScannerImpl.java:1029)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchDouble(ScannerImpl.java:1011)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396)", "org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)", "org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:557)", "org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)", "org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:167)", "org.jruby.ext.psych.PsychParser.parse(PsychParser.java:186)", "org.jruby.ext.psych.PsychParser$INVOKER$i$parse.call(PsychParser$INVOKER$i$parse.gen)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:193)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.invokeOther3:parse(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:377)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.RUBY$method$parse_stream$0(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:377)", "java.lang.invoke.LambdaForm$DMH/71587369.invokeStatic_L7_L(LambdaForm$DMH)", "java.lang.invoke.LambdaForm$MH/1952473604.invokeExact_MT(LambdaForm$MH)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:113)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:198)", "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:203)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:209)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.invokeOther1:parse_stream(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:325)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.RUBY$method$parse$0(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:325)", "java.lang.invoke.LambdaForm$DMH/71587369.invokeStatic_L7_L(LambdaForm$DMH)", "java.lang.invoke.LambdaForm$MH/1952473604.invokeExact_MT(LambdaForm$MH)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:126)"], :thread=>"#"} ``` `[2018-05-29T11:34:40,177][ERROR][org.logstash.Logstash] java.lang.OutOfMemoryError: Java heap space` Here is my jvm config for logstash ``` Xms represents the initial size of total heap space Xmx represents the maximum size of total heap space -Xms4g -Xmx4g ``` Here is my jvm for elasticsearch ``` # Xms represents the initial size of total heap space # Xmx represents the maximum size of total heap space -Xms8g -Xmx8g ``` ---
**Author:** ![dalamar666](https://avatars.discourse-cdn.com/v4/letter/d/e9bcb4/32.png) [@dalamar666](https://discuss.elastic.co/u/dalamar666)\ **Post date:** [May 29, 2018, 4:08pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221/4 "2018-05-29T16:08:20Z")
I am also finding that there are only 3 instances of Java starting and I have 5 pipelines so I should have more instances of Java. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 26, 2018, 4:13pm UTC](https://discuss.elastic.co/t/limit-to-number-of-pipelines/133221/5 "2018-06-26T16:13:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.