RabbitMQ plugin filebeat questions

Elastic Stack Beats
1

I have a filebeat configured to pull data from a Rabbitmq with its module. Now on the official website it only gives var.paths:

Question, if my Rabbitmq has a username and password and is on a host (ip or url) how do I attack to get there?

####Filebeat####
- module: rabbitmq
  log:
    enabled: true
    var.paths: ["/var/log/rabbitmq/*.log*"]

But where i put user, password, ip or same

2

Hi,

probably because the RabbitMQ module for Filebeat is designed to ingest RabbitMQ logs, not to connect to the RabbitMQ service directly.

Regards

3

As mentioned in the documentation this module is used to get logs from the RabbitMQ service, not data in RabbitMQ.

This is the module for parsing RabbitMQ log files It will only support RabbitMQ default i.e RFC 3339 timestamp format using TIMESTAMP_ISO8601.

To get data from RabbitMQ you would need to use Logstash and the rabbitmq input.

4

mmmm let's see I explain the situation a little.

I have the filebeat with a Redis module and another Rabbitmq module, I want to get to those logs that the Rabbit leaves in /var/logs, so what would be the scenario?

I have a metricbeat that gives me information from those two, but for the filebeat do I need the logstash? I have it mounted on a k3s (kubernetes) so if the filebeat is on the same environment it does not need a host/ip

Basically I need to be able to see the Rabbitmq logs in Kibana, for this I know that the filebeat is for the logs, but for example Rabbitmq has a username and password, don't I have to pass that on to it?

5

If you want to get the logs that the RabbitMQ server writes inside /var/log/rabbitmq then there is no need for username and password and you can use the filebeat module.

The username and password are used to access a queue in RabbitMQ and get data from the queue.

This is a different thing, filebeat can get the logs from the service, but cannot get data from rabbitmq queues.

6

that it, i need de logs from the service with filebeat.

so, the config it ok?
- module: rabbitmq log: enabled: true var.paths: ["/var/log/rabbitmq/*.log*"]

or i need to put this:

- module: rabbitmq
  log:
    enabled: true
    paths: ["/var/log/rabbitmq/*.log*"]
7

i have k3s and in my kibana.yaml i have this:

apiVersion: v1
data:
  filebeat.yml: |
    filebeat.modules:
    - module: rabbitmq
      log:
        enabled: true
        var.paths: ["/var/log/containers/m*rabbit*.log", "/var/log/containers/*rabbit*.log", "-/var/log/rabbitmq/*.log"]
    - module: redis
      log:
        enabled: true
        var.paths: ["/var/log/redis/redis-server.log*"]
      slowlog:
        enabled: true
        var.hosts: ["redis.svc.cluster.local:6379"]
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          matchers:
          - logs_path:
              logs_path: "/var/log/containers/"
    - type: log
      enabled: true
      paths:
        - /var/log/containers/*rabbitmq*/*.log
        - /var/log/containers/m*rabbit*.log
      fields:
        log_type: rabbitmq
      fields_under_root: true
    filebeat.inputs:
    - type: filestream
      id: rabbitmq
      paths:
        - /var/log/containers/*rabbitmq*/*.log
        - /var/log/containers/m*rabbit*.log
        - /var/log/rabbitmq/*.log

any error?

8

No idea, I do not use Kubernetes.

But I don't think this is correct.

Is the /var/log/rabbitmq path available inside the filebeat pod?

Where is your RabbitMQ service running?

9

this path no, i have path containers who have a rabbitmq

so i need to see this in kibana
imagen

rabbitmq.log.pid but i dont know how to configure

and this i can found

imagen

imagen
imagen1742×152 7.78 KB

the logs is in /var/log/pods/ and i have the logs in this directory for rabbitmq

10

Solved:

    - type: container
      paths:
        - /var/log/containers/*rabbitmq*/*.log
        - /var/log/pods/*rabbitmq*/*.log
      processors:
      - add_kubernetes_metadata:
          host: ${NODE_NAME}
          in_cluster: true
          matchers:
          - logs_path:
                - /var/log/pods/*rabbitmq*/
                - /var/log/containers/*rabbitmq*/
                - /var/log/pods/rabbitmq*/
      - dissect:
          tokenizer: "%{[@metadata][log]} %{rabbitmq.log.pid} %{[@metadata][message]}"
          field: "message"
          target_prefix: "rabbitmq"