LOGSTASH AND PIPELINE

Elastic Stack Logstash
1

Hi,

I am using distributor approch for two usecases.My config is:

config/pipelines.yml

pipeline.id: beats2
config.string: |
input {
file {
path => "/opt/webmis/ELKFMDOFMSDKLFJAVA/GA/guyubdjsanbticleGlobal/.txt"
start_position => "beginning"
add_field => [ "log_type","FA-SAP" ]
}
file {
path => "/opt/webmis/ELNFJKDHFKDN_JAVA/YA/IDoc_OutboundArticleLocal/.txt"
start_position => "beginning"
add_field => [ "log_type","LA-SOP" ]
}
}
output {
if [log_type] == FA-SAP {
pipeline { send_to => beats9}
} else if [log_type] == LA-SOP{
pipeline { send_to => beats10}
}
}

i am getting error in logstash logs as:
[2018-11-28T10:15:46,341][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:beats2, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, input, filter, output at line 2, column 1 (byte 25) after # config/pipelines.yml\r\n", :backtrace=>["/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:42:in compile_imperative'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:50:incompile_graph'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:12:in block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:11:in compile_sources'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/pipeline.rb:51:ininitialize'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/pipeline.rb:169:in initialize'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/pipeline_action/create.rb:40:inexecute'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:141:inwith_pipelines'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:ineach'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:166:inblock in converge_state_and_update'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:164:inconverge_state_and_update'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:105:in block in execute'", "/opt/webmis/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:ininterval'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:94:in execute'", "/opt/webmis/logstash-6.2.4/logstash-core/lib/logstash/runner.rb:348:inblock in execute'", "/opt/webmis/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}

Please help.
I am working with centralized pipeline.

2

You should follow this format:

- pipeline.id: my-pipeline_1
  path.config: "/etc/path/to/p1.config"
  pipeline.workers: 3
- pipeline.id: my-other-pipeline
  path.config: "/etc/different/path/p2.cfg"
  queue.type: persisted

Also you have a missing hyphon in your configuration at the start before "pipeline.id"

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.