Logstash output by condition

Elastic Stack Logstash
1

Hi Guys,
I setup logstash with influxdb plugin, and can send metric to influxdb successfully. But now I meet question with output by condition.
run two filebeat instance onto two pc to capture two different log files, I want send to same logstash server, logstash process these message by different signal (etc, log name) and sent same influxdb by differentment measurement( measurement is influxdb element). I test if but not working for me. can you give me any suggestion? thanks

2

Hi @tbs575,

Using the conditional operator, similar to this thread is probably the best approach. It sounds like you've have issues with that approach. Can you give a snippet of what you've tried and if you received a particular error?

3

part from logstash.conf, I want output same server (influxdb), just different measurement by condition, and met error.

output {
  influxdb {
    host => ["10.200.101.18"]
    id => "logstash_id"
    user =>  "logstash"
    password => "logstash"
    if [message] =~ "destination-port" {
      measurement => "energy-log"
    } else {
      measurement => "tttt-log"
    }
    send_as_tags => ["[host][name]"]
    db => "test"
    exclude_fields => ["original"]
    use_event_fields_for_data_points => false
    data_points => {
       "source_ip"=> "%{source_ip}" "source_port"=> "%{source_port}" "dest_ip"=> "%{dest_ip}" "dest_port"=> "%{dest_port}"
    }
  }
}

logstash-influxdb  | [2023-07-17T01:19:25,713][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 25, column 8 (byte 527) after output {\n  influxdb {\n    host => [\"10.200.101.18\"]\n    id => \"logstash_id\"  \n    user =>  \"logstash\"\n    password => \"logstash\"\n    if ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:911:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}
4

You cannot use a conditional inside the output, you need the use two outputs

output {
     if [message] =~ "destination-port" {
         influxdb {
             host => ["10.200.101.18"]
             ...
             measurement => "energy-log"
             ...
        }
    } else {
         influxdb {
             host => ["10.200.101.18"]
             ...
             measurement => "tttt-log"
             ...
        }     
    }
}

The very first post I made on this site was about me making exactly this mistake :rofl:

1 Like
5

thanks, fixed. Can I use pipelines? like this link (Multiple Pipelines | Logstash Reference [8.8] | Elastic), using different file by condition, thanks

6

If I understand your ask correctly then yes, the distributor pattern of pipeline-to-pipeline communication uses conditionals to route to different pipelines.

7

yes, thanks

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.