Error wen put pipeline line on conf file (version 8.7)

[jefin_dark (Jefferson Vasconcelos)]

Hello, i cant start service, because the logstash bring me this error when i put this lines in conf file.

The given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", "input", "filter", "output" at line 1, column 1

my conf file is

pipeline {
  pipeline_id => "mikrotik"
 }
  
input {
  udp {
    port => 5514
    codec => plain
  }
}

filter {
  grok {
    match => {
      "message" => "<%{POSINT:priority}>%{MONTH:month} %{MONTHDAY:day} %{TIME:time} %{DATA:hostname} %{WORD:event_type} %{GREEDYDATA:message_data}"
    }
  }

  mutate {
    convert => { "priority" => "integer" }
  }

  if [event_type] == "user" {
    grok {
      match => {
        "message_data" => "admin logged in from %{IP:source_ip} via %{WORD:login_method}"
      }
    }
  }

  if [event_type] == "filter" {
    grok {
      match => {
        "message_data" => "rule %{WORD:rule_action} by %{DATA:rule_modifier}"
      }
    }
  }
}

output {
  elasticsearch {
    hosts => ["https://192.168.12.109:9200"]
    index => "mikrotik_log-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "xxxxxxxxxxxxxxx"
    ssl => true
    cacert => "/etc/logstash/http_ca.crt"
  }
}

Someone can help ?

[stephenb (Stephen Brown)]

Hi @jefin_dark Welcome to the community.

Structure of a pipeline | Logstash Reference [8.11] | Elastic

Take out this, I'm not sure where you got that from. That is not how you name a pipeline

@jefin_dark

pipeline {
  pipeline_id => "mikrotik"
 }

[jefin_dark (Jefferson Vasconcelos)]

ok, if i take it out it will work.
But if I add another conf file with another port, it will send duplicate data in the indexes inside the ELK.

thats other conf

input {
  beats {
    port => 5044
  }
}

filter {
  grok {
    match => {
      "[event_data][Data]" => [
        "Subject:\s*Security ID:\s*%{DATA:security_id}",
        "Subject:\s*Account Name:\s*%{DATA:account_name}",
        "Subject:\s*Account Domain:\s*%{DATA:account_domain}",
        "Subject:\s*Logon ID:\s*%{DATA:logon_id}",
        "Object:\s*Object Server:\s*%{DATA:object_server}",
        "Object:\s*Object Type:\s*%{DATA:object_type}",
        "Object:\s*Object Name:\s*%{DATA:object_name}",
        "Object:\s*Handle ID:\s*%{DATA:handle_id}",
        "Access:\s*Accesses:\s*%{DATA:accesses}",
        "Access:\s*Access Mask:\s*%{DATA:access_mask}",
        "Access:\s*Privileges:\s*%{DATA:privileges}"
      ]
    }
  }
}

output {
  elasticsearch {
    hosts => ["https://192.168.12.109:9200"]
    index => "srvvmfs01_log-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "*****************"
    ssl => true
    cacert => "/etc/logstash/http_ca.crt"
  }
}

Could it be that I'm doing something wrong?

Can you help me ?

[stephenb (Stephen Brown)]

I think you should read a little in the docs

This is how to run multiple independent pipelines naming them kind of like what you are trying to do... But do it in the correct place

Multiple Pipelines | Logstash Reference [8.7] | Elastic

[jefin_dark (Jefferson Vasconcelos)]

I've done this before.
I'm going to mark it as done and open another topic and explain better what the error I'm facing.

Thank you for your help and attention

[stephenb (Stephen Brown)]

Yeah you can open another topic or just show us all the configuration and what the actual error is.

If you put more than one configuration file in a directory and don't use pipeline.yml

The two configuration files get concatenated together so all events go to all outputs.

But you absolutely can put tags on the inputs and then use those tags to conditionally direct to the output you want.

That is a very common configuration

You could do that in one big conf file or two separate conf.

Tell us what You're trying to accomplish and we can certainly help.

If you want to open another topic that's fine too...