Filebeat logs not displaying to Kibana

Elastic Stack Beats
1

Filebeat YAML configuration files automated by Ansible playbook have been installed on two virtual machines (Web1 & Web2). I can SSH into them and find filebeat systemctl service is running (active). But no log data is displaying on Kibana.

image
image1905×1172 83.8 KB

Strangely though the same configuration for metric beat is displaying system metrics through Kibana. (Also Web1 & 2). I don't understand how metric data is coming through but system log data is not.

image
image1888×1210 310 KB

2

Welcome to our community! :smiley:

What do your Filebeat logs show?
What version are you on?
What does your config look like?

3

Hi Warkolm,

I havent tried to view the filebeat logs yet in the webservers. Which directory should I cd into and log file should I cat to provide that data? Also for additional reference, The two webservers are running DVWA and DVWA wasn't running correctly until i disabled Apache 2 on both webservers, not sure if that would be interfering with the kibana display for filebeat but wanted to let you know.

Filebeat version: 7.4.0-amd54.deb

Filebeat playbook:

---
- name: installing and launching filebeat
  hosts: webservers
  become: yes
  tasks:

    - name: download filebeat deb
      command: curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.4.0-amd64.deb

    - name: install filebeat deb
      command: dpkg -i filebeat-7.4.0-amd64.deb

    - name: drop in filebeat.yml
      copy:
        src: /etc/filebeat/filebeat-config.yml
        dest: /etc/filebeat/filebeat.yml

    - name: enable and configure system module
      command: filebeat modules enable system

    - name: setup filebeat
      command: filebeat setup

    - name: start filebeat service
      command: service filebeat start

    - name: enable service filebeat on boot
      systemd:
         name: filebeat
         enabled: yes

Host Configuration:

# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
#   - Comments begin with the '#' character
#   - Blank lines are ignored
#   - Groups of hosts are delimited by [header] elements
#   - You can enter hostnames or ip addresses
#   - A hostname/ip can be a member of multiple groups

# Ex 1: Ungrouped hosts, specify before any group headers.

#green.example.com
#blue.example.com
#192.168.100.1
#192.168.100.10

# Ex 2: A collection of hosts belonging to the 'webservers' group

[webservers]
#alpha.example.org
#beta.example.org
#192.168.1.100
#192.168.1.110
10.0.0.7 ansible_python_interpreter=/usr/bin/python3
10.0.0.8 ansible_python_interpreter=/usr/bin/python3

[elk]
10.1.0.5 ansible_python_interpreter=/usr/bin/python3

# If you have multiple hosts following a pattern you can specify
# them like this:

#www[001:006].example.com

# Ex 3: A collection of database servers in the 'dbservers' group

#[dbservers]
#
#db01.intranet.mydomain.net
#db02.intranet.mydomain.net
#10.25.1.56
#10.25.1.57

# Here's another example of host ranges, this time there are no
# leading 0s:

#db-[99:101]-node.example.com
4

Filebeat Configuration File:

#-------------------------- Elasticsearch output -------------------------------
output.elasticsearch:

  # Boolean flag to enable or disable the output module.
  #enabled: true

  # Array of hosts to connect to.
  # Scheme and port can be left out and will be set to the default (http and 9200)
  # In case you specify and additional path, the scheme is required: http://localhost:9200/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
  hosts: ["10.1.0.5:9200"]


#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
  host: "10.1.0.5:5601"
  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  #host: "localhost:5601"
5

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile:

6

Thanks for the instructions Warkolm. I hope that reads better.

7

Great, thank you!

Probably /var/log/filebeat/filebeat.log.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.