[Filebeat System] Syslog dashboard ECS showing no results found

**filebeat.yml file **

**###################### Filebeat Configuration Example #########################**

**# This file is an example configuration file highlighting only the most common**
**# options. The filebeat.reference.yml file from the same directory contains all the**
**# supported options with more comments. You can use it as a reference.**
**#**
**# You can find the full configuration reference here:**
**# https://www.elastic.co/guide/en/beats/filebeat/index.html**

**# For more available modules and options, please see the filebeat.reference.yml sample**
**# configuration file.**

**# ============================== Filebeat inputs ===============================**

**filebeat.inputs:**

**# Each - is an input. Most options can be set at the input level, so**
**# you can use different inputs for various configurations.**
**# Below are the input specific configurations.**

**- type: log**

**  # Change to true to enable this input configuration.**
**  enabled: true**

**  # Paths that should be crawled and fetched. Glob based paths.**
**  paths:**
**    #- /var/log/messages**
**    - /var/log/secure**
**    - /var/log/cron**
**    #- c:\programdata\elasticsearch\logs\***

**  # Exclude lines. A list of regular expressions to match. It drops the lines that are**
**  # matching any regular expression from the list.**
**  #exclude_lines: ['^DBG']**

**  # Include lines. A list of regular expressions to match. It exports the lines that are**
**  # matching any regular expression from the list.**
**  #include_lines: ['^ERR', '^WARN']**

**  # Exclude files. A list of regular expressions to match. Filebeat drops the files that**
**  # are matching any regular expression from the list. By default, no files are dropped.**
**  #exclude_files: ['.gz$']**

**  # Optional additional fields. These fields can be freely picked**
**  # to add additional information to the crawled log files for filtering**
**  #fields:**
**  #  level: debug**
**  #  review: 1**

**  ### Multiline options**

**  # Multiline can be used for log messages spanning multiple lines. This is common**
**  # for Java Stack Traces or C-Line Continuation**

**  # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [**
**  #multiline.pattern: ^\[**

**  # Defines if the pattern set under pattern should be negated or not. Default is false.**
**  #multiline.negate: false**

**  # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern**
**  # that was (not) matched before or after or as long as a pattern is not matched based on negate.**
**  # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash**
**  #multiline.match: after**

**# filestream is an experimental input. It is going to replace log input in the future.**
**- type: filestream**

**  # Change to true to enable this input configuration.**
**  enabled: false**

**  # Paths that should be crawled and fetched. Glob based paths.**
**  paths:**
**    - /var/log/*.log**
**    #- c:\programdata\elasticsearch\logs\***

**  # Exclude lines. A list of regular expressions to match. It drops the lines that are**
**  # matching any regular expression from the list.**
**  #exclude_lines: ['^DBG']**

**  # Include lines. A list of regular expressions to match. It exports the lines that are**
**  # matching any regular expression from the list.**
**  #include_lines: ['^ERR', '^WARN']**

**  # Exclude files. A list of regular expressions to match. Filebeat drops the files that**
**  # are matching any regular expression from the list. By default, no files are dropped.**
**  #prospector.scanner.exclude_files: ['.gz$']**

**  # Optional additional fields. These fields can be freely picked**
**  # to add additional information to the crawled log files for filtering**
**  #fields:**
**  #  level: debug**
**  #  review: 1**

**# ============================== Filebeat modules ==============================**

**filebeat.config.modules:**
**  # Glob pattern for configuration loading**
**  path: ${path.config}/modules.d/*.yml**

**  # Set to true to enable config reloading**
**  reload.enabled: true**

**  # Period on which files under path should be checked for changes**
**  #reload.period: 10s**

**# ======================= Elasticsearch template setting =======================**

**setup.template.settings:**
**  index.number_of_shards: 1**
**  #index.codec: best_compression**
**  #_source.enabled: false**


**# ================================== General ===================================**

**# The name of the shipper that publishes the network data. It can be used to group**
**# all the transactions sent by a single shipper in the web interface.**
**#name:**

**# The tags of the shipper are included in their own field with each**
**# transaction published.**
**#tags: ["service-X", "web-tier"]**

**# Optional fields that you can specify to add additional information to the**
**# output.**
**#fields:**
**#  env: staging**

**# ================================= Dashboards =================================**
**# These settings control loading the sample dashboards to the Kibana index. Loading**
**# the dashboards is disabled by default and can be enabled either by setting the**
**# options here or by using the `setup` command.**
**#setup.dashboards.enabled: true**
**dashboards.enabled: true**
**# The URL from where to download the dashboards archive. By default this URL**
**# has a value which is computed based on the Beat name and version. For released**
**# versions, this URL points to the dashboard archive on the artifacts.elastic.co**
**# website.**
**#setup.dashboards.url:**

**# =================================== Kibana ===================================**

**# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.**
**# This requires a Kibana endpoint configuration.**
**setup.kibana:**

**  # Kibana Host**
**  # Scheme and port can be left out and will be set to the default (http and 5601)**
**  # In case you specify and additional path, the scheme is required: http://localhost:5601/path**
**  # IPv6 addresses should always be defined as: https://[x:x:x:x::1]:5601**
**  host: "x.x.x.x:5601"**

**  # Kibana Space ID**
**  # ID of the Kibana Space into which the dashboards should be loaded. By default,**
**  # the Default Space will be used.**
**  #space.id:**

**# =============================== Elastic Cloud ================================**

**# These settings simplify using Filebeat with the Elastic Cloud (https://cloud.elastic.co/).**

**# The cloud.id setting overwrites the `output.elasticsearch.hosts` and**
**# `setup.kibana.host` options.**
**# You can find the `cloud.id` in the Elastic Cloud web UI.**
**#cloud.id:**

**# The cloud.auth setting overwrites the `output.elasticsearch.username` and**
**# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.**
**#cloud.auth:**

**# ================================== Outputs ===================================**

**# Configure what output to use when sending the data collected by the beat.**

**# ---------------------------- Elasticsearch Output ----------------------------**
**output.elasticsearch:**
**  # Array of hosts to connect to.**
**  #hosts: ["localhost:9200"]**
**  hosts: ["http://x.x.x.x:9200"]**
**  username: "elastic"**
**  password: "elastic"**
**setup.kibana:**
**  host: "http://x.x.x.x:5601"**

**  # Protocol - either `http` (default) or `https`.**
**  protocol: "http"**

**  # Authentication credentials - either API key or username/password.**
**  #api_key: "id:api_key"**
**  #username: "elastic"**
**  #password: "changeme"**

**# ------------------------------ Logstash Output -------------------------------**
**#output.logstash:**
**  # The Logstash hosts**
**  #hosts: ["localhost:5044"]**

**  # Optional SSL. By default is off.**
**  # List of root certificates for HTTPS server verifications**
**  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]**

**  # Certificate for SSL client authentication**
**  #ssl.certificate: "/etc/pki/client/cert.pem"**

**  # Client Certificate Key**
**  #ssl.key: "/etc/pki/client/cert.key"**

**# ================================= Processors =================================**
**processors:**
**  - add_host_metadata:**
**      when.not.contains.tags: forwarded**
**  - add_cloud_metadata: ~**
**  - add_docker_metadata: ~**
**  - add_kubernetes_metadata: ~**

**# ================================== Logging ===================================**

**# Sets log level. The default log level is info.**
**# Available log levels are: error, warning, info, debug**
**#logging.level: debug**

**# At debug level, you can selectively enable logging only for some components.**
**# To enable all selectors use ["*"]. Examples of other selectors are "beat",**
**# "publisher", "service".**
**#logging.selectors: ["*"]**

**# ============================= X-Pack Monitoring ==============================**
**# Filebeat can export internal metrics to a central Elasticsearch monitoring**
**# cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The**
**# reporting is disabled by default.**

**# Set to true to enable the monitoring reporter.**
**#monitoring.enabled: false**

**# Sets the UUID of the Elasticsearch cluster under which monitoring data for this**
**# Filebeat instance will appear in the Stack Monitoring UI. If output.elasticsearch**
**# is enabled, the UUID is derived from the Elasticsearch cluster referenced by output.elasticsearch.**
**#monitoring.cluster_uuid:**

**# Uncomment to send the metrics to Elasticsearch. Most settings from the**
**# Elasticsearch output are accepted here as well.**
**# Note that the settings should point to your Elasticsearch *monitoring* cluster.**
**# Any setting that is not set is automatically inherited from the Elasticsearch**
**# output configuration, so if you have the Elasticsearch output configured such**
**# that it is pointing to your Elasticsearch monitoring cluster, you can simply**
**# uncomment the following line.**
**#monitoring.elasticsearch:**

**# ============================== Instrumentation ===============================**

**# Instrumentation support for the filebeat.**
**#instrumentation:**
**    # Set to true to enable instrumentation of filebeat.**
**    #enabled: false**

**    # Environment in which filebeat is running on (eg: staging, production, etc.)**
**    #environment: ""**

**    # APM Server hosts to report instrumentation results to.**
**    #hosts:**
**    #  - http://localhost:8200**

**    # API Key for the APM Server(s).**
**    # If api_key is set then secret_token will be ignored.**
**    #api_key:**

**    # Secret token for the APM Server(s).**
**    #secret_token:**


**# ================================= Migration ==================================**

**# This allows to enable 6.7 migration aliases**
**#migration.6_to_7.enabled: true**
**setup.ilm.overwrite: true**

**setup.ilm.enabled: auto**
**setup.ilm.rollover_alias: "filebeat-linuxclient"**
**setup.ilm.pattern: "{now/d}-000001"**
**setup.template.overwrite: true**

**}}}}}**



**system file**
**{**

**[root@gc-bcmt-cwe1 ~]# cat /etc/filebeat/modules.d/system**
**# Module: system**
**# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.11/filebeat-module-system.html**

**- module: system**
**  # Syslog**
**  syslog:**
**    enabled: true**

**    # Set custom paths for the log files. If left empty,**
**    # Filebeat will choose the paths depending on your OS.**
**    #var.paths: ["/var/log/yum.log"]**
**    var.paths: ["/var/log/secure"]**

**  # Authorization logs**
**  auth:**
**    enabled: true**

**    # Set custom paths for the log files. If left empty,**
**    # Filebeat will choose the paths depending on your OS.**
**    var.paths: /var/log/secure**

** input:**
**      multiline.pattern: '\['**
**      multiline.negate: false**
**      multiline.match: after**
**[root@gc-bcmt-cwe1 ~]#**

**}}}**

**Can you please let me know, if any thing wrongly configured in yml file.**
**When loaded filebeat with systemmodule through kibana===>adddata===>security===>systemlogs**

**syslog,sudo_command, ssh logins, new user creation logs not coming and showing as no results found. can you please help me here**

Are you seeing the data in the index/discover page? You also have a regular file input reading from the same path var/log/secure, that data will probably not show up in the module dashboard because it won't have the correct fields.

Thanks a lot for your support


Are you seeing the data in the index/discover page?
ShabariNath : Yes. Attached snap shot.

You also have a regular file input reading from the same path var/log/secure , that data will probably not show up in the module dashboard because it won't have the correct fields
ShabariNath : Can you please share some sample configurations which was working for you, so that it can be helpful for me

pls check the path.

it may b /var/log/messages for syslog

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.