Syslog client doesn't send to Logstash

Elastic Stack Logstash
1

Hello,

Hello,

I would like to get some advice for this configuration: Send rsyslog to my logstash.
I tested both configurations without success.
One moment it worked after restarting the pipeline nothing.

$ModLoad imudp
$UDPServerRun 5000
 #module(load="imudp")
#input(type="imudp" port="5000")
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 5000
#module(load=”imptcp”)
#input(type=”imptcp” port=”5000″)
*.* @xxx:5000

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch {
    hosts => ["https://xxx:9200", "https://xx:9200", "https://xx:9200"]
    ssl => true
    ssl_certificate_verification => true
    keystore => "/etc/logstash/certs/xxxxxx"
    keystore_password => "${KEY}"
    cacert => "/etc/logstash/certs/xxxx
    api_key => "xxx"
    index => "syslog-%{+YYYY.MM.dd}"
}
}

syslog_message"=>"Started System Logging Service."}], :error=>{"type"=>"security_exception", "reason"=>"action [indices:admin/auto_create] is unauthorized for API key id [JoyBgoUBDSlOApYFv-1N] of user [elastic] on indices [syslog-2023.01.06], this action is granted by the index privileges [auto_configure,create_index,manage,all]"}}

yet I think I have registered the necessary rights during the creation :

{
  "name": "syslog", 
  "role_descriptors": {
    "logstash_writer": { 
      "cluster": ["all"],
      "index": [
        {
          "names": ["syslog-*"],
          "privileges": ["auto_configure", "create_index", "manage", "all"]
        }
      ]
    }
  }
}
'
2

Hi,

The error message is pretty much enough to understand that you have a permission problem on the user of your cluster !

3

Hi,
I understood that the problem came from the user.
But I don't know what right he needs more than :

"auto_configure", "create_index", "manage", "all"

But my primary question, is about sending log from a syslog client, my configuration is present at the beginning of the conversation

At this moment I don't receive any log yet logstash is listening:

Starting tcp input listener {:address=>"0.0.0.0:5000", :ssl_enable=>false}
Starting UDP listener {:address=>"0.0.0.0:5000"}
4

You might not be able to receive any logs since the user is not allowed to index.

Try stdout input without elastic from logstash.

5

After several unsuccessful attempts but I still manage to get an error feedback.

It seems that it is related to
to "names" : ["syslog-"] in the api key which doesn't match with the set of logs sent as for example logs-generic-default in the error return
How to cover a larger part of the log names?
because ["syslog-"] in ["*"] not work.

and in this sentence : unauthorized for API key id of user [elastic] on indices `
I'm not sure why the elastic user is concerned.

Thanks for your insights

Retrying individual bulk actions that failed or were rejected by the previous bulk request {:count=>2}
[2023-01-06T18:04:57,182][INFO ][logstash.outputs.elasticsearch][main][60174f377f2619fb21f770e177514cb0f0a9c5a0c4d8c73131c1ff3e09691cfc] Retrying failed action {:status=>403, :action=>["create", {:_id=>nil, :_index=>"**logs-generic-default**", :routing=>nil}, {"@version"=>"1", "syslog_pid"=>"1", "syslog_timestamp"=>"Jan  6 18:04:40", "syslog_hostname"=>"xxxx, "@timestamp"=>2023-01-06T17:04:40.000Z, "syslog_program"=>"systemd", "type"=>"syslog", "message"=>"<30>Jan  6 18:04:40 xxxx systemd[1]: Stopping System Logging Service...", "syslog_message"=>"Stopping System Logging Service...", "host"=>{"ip"=>"xxxx"}, "received_at"=>"2023-01-06T17:04:51.028557682Z", "received_from"=>"{\"ip\":\"xxxx\"}", "event"=>{"original"=>"<30> xxxx systemd[1]: Stopping System Logging Service..."}, "data_stream"=>{"type"=>"logs", "dataset"=>"generic", "namespace"=>"default"}}], :error=>{"type"=>"security_exception", "reason"=>"action [indices:data/write/bulk[s]] is unauthorized for API key id [xxxxx] of user [elastic] on indices [logs-generic-default,.ds-logs-generic-default-2023.01.06-000001], this action is granted by the index privileges [create_doc,create,delete,index,write,all]"}}

{
  "name": "syslog", 
  "role_descriptors": {
    "logstash_writer": { 
      "cluster": ["all"],
      "index": [
        {
          "names": ["syslog-*"],
          "privileges": ["auto_configure", "create_index", "manage", "all"]
        }
      ]
    }
  }
}
'
6

Hello,

You'll need to review the permissions, i know it can be touchy but 403 still stand for a unauthorized error type.

You might want to try using the defaults groups/perms for logstash users.

7

One Thought... something I have run into..

What user did you create the API with... an API key can not have greater permissions than the user that created it... the bad/confusing part is the key can be created but it will not actually work...

So if the user you used to create the API did not have all those permissions ... the API will not actually have that privilege

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.