Limit to number of pipelines

[dalamar666]

Hi,

I have 5 pipelines on my machine. I have 8 CPU's and enough RAM and disk space for them. Only 3 of the pipelines are reporting data and creating indices. I have verified all of my configurations to make sure I did not miss anything and I have checked with tshark to make sure that I was getting data.

Logstash 6.2 Unbuntu Server 16.04.

[Christian_Dahlqvist]

What do the pipelines look like? Do they have any identical inputs that could cause problems? Is there anything in the Logstash logs?

[dalamar666]

They all have identical inputs. The inputs are for netflow and they are all using different ports to keep things separate. The logs show things appearing to be working as normal however, my indices are not increasing with data. I do see this error

[2018-05-29T11:34:39,815][ERROR][logstash.pipeline        ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {:pipeline_id=>"edpnc", "exception"=>"Java heap space", "backtrace"=>["java.util.Arrays.copyOfRange(Arrays.java:3664)", "java.lang.String.<init>(String.java:207)", "java.lang.StringBuilder.toString(StringBuilder.java:407)", "org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:247)", "org.yaml.snakeyaml.reader.StreamReader.peek(StreamReader.java:160)", "org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalarNonSpaces(ScannerImpl.java:1865)", "org.yaml.snakeyaml.scanner.ScannerImpl.scanFlowScalar(ScannerImpl.java:1845)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchFlowScalar(ScannerImpl.java:1029)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchDouble(ScannerImpl.java:1011)", "org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:396)", "org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)", "org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:557)", "org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)", "org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:167)", "org.jruby.ext.psych.PsychParser.parse(PsychParser.java:186)", "org.jruby.ext.psych.PsychParser$INVOKER$i$parse.call(PsychParser$INVOKER$i$parse.gen)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:193)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.invokeOther3:parse(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:377)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.RUBY$method$parse_stream$0(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:377)", "java.lang.invoke.LambdaForm$DMH/71587369.invokeStatic_L7_L(LambdaForm$DMH)", "java.lang.invoke.LambdaForm$MH/1952473604.invokeExact_MT(LambdaForm$MH)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:113)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:198)", "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:203)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:209)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.invokeOther1:parse_stream(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:325)", "uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.psych.RUBY$method$parse$0(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/psych.rb:325)", "java.lang.invoke.LambdaForm$DMH/71587369.invokeStatic_L7_L(LambdaForm$DMH)", "java.lang.invoke.LambdaForm$MH/1952473604.invokeExact_MT(LambdaForm$MH)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:126)"], :thread=>"#<Thread:0x7d6f54b5@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:247 sleep>"}

[2018-05-29T11:34:40,177][ERROR][org.logstash.Logstash ] java.lang.OutOfMemoryError: Java heap space

Here is my jvm config for logstash

Xms represents the initial size of total heap space
Xmx represents the maximum size of total heap space

-Xms4g
-Xmx4g

Here is my jvm for elasticsearch

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms8g
-Xmx8g

[dalamar666]

I am also finding that there are only 3 instances of Java starting and I have 5 pipelines so I should have more instances of Java.