Logstash showing error when executed from python but executes from terminal

Elastic Stack Logstash
1

I'm new to ELK stack and any help would be appriciated
I have tried both os and subprocess both are giving same error

This is getting executed from terminal
`

/usr/share/logstash/bin/logstash -f /root/folder1/folder2/demo.conf

`

but the Python code:

os.system("/usr/share/logstash/bin/logstash -f /root/folder1/folder2/demo.conf")

similar thing was working on my windows machine but on server it's giving error

Can see the error output below:

[ERROR] 2022-08-05 10:43:45.366 [Ruby-0-Thread-13: :1] javapipeline - Worker loop initialization error {:pipeline_id=>"main", :error=>"Missing Filter End Vertex", :exception=>Java::JavaLang::IllegalStateException, :stacktrace=>"org.logstash.config.ir.CompiledPipeline$CompiledExecution.lambda$compileFilters$1(org/logstash/config/ir/CompiledPipeline.java:384)\njava.util.Optional.orElseThrow(java/util/Optional.java:408)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.compileFilters(org/logstash/config/ir/CompiledPipeline.java:384)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.(org/logstash/config/ir/CompiledPipeline.java:363)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.(org/logstash/config/ir/CompiledPipeline.java:322)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/CompiledPipeline.java:154)\norg.logstash.execution.WorkerLoop.(org/logstash/execution/WorkerLoop.java:67)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:62)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:490)\norg.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:253)\norg.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:939)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)\nRUBY.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:578)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:285)\norg.jruby.RubyProc.call(org/jruby/RubyProc.java:318)\njava.lang.Thread.run(java/lang/Thread.java:829)", :thread=>"#<Thread:0x5b2c8287 sleep>"}

2

What does that file look like on the server that is producing that error?

3

Here is the demo.conf file. Can there be an issue with access to this file or unable to read from this file?

input 
{
    file 
    {   
        path => "path/demo.json"
        mode => "read"
        file_completed_action => 'log'
        file_completed_log_path => 'path/writen_files.log'
        exit_after_read => 'true'
        sincedb_path => "/dev/null"
        exclude => "*.gz"
    }
}

filter {
    json {
        source => "message" 
    }

    mutate {
        remove_field => ["host","path","@version","message"]
    }
}

output {
    elasticsearch { 
        codec => json
        hosts => ["http://host:9200/"]
        index => "req_idx"
    }
    stdout {}
}
4

Neither of those problems would cause that message. Not sure what is wrong.

5

Executing it from a different script fixed it. (External python file)

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.