Able to configure Filebeat and index in Kibana, but no data is appearing

Elastic Stack Beats
1

I was able to configure the Filebeat yaml and index it in Kibana, but no data is appearing. Service is running but no information is displaying in my discover tab or in my filebeat iis dashboard.

Below is my filebeat.yml

#=========================== Filebeat inputs =============================

filebeat.inputs:
- type: log
  enabled: true
  paths:
  - C:\intepub\logs\LogFiles\*\*
    #- c:\programdata\elasticsearch\logs\*

  # Exclude lines. A list of regular expressions to match. It drops the lines that are
  # matching any regular expression from the list.
  #exclude_lines: ['^DBG']

  # Include lines. A list of regular expressions to match. It exports the lines that are
  # matching any regular expression from the list.
  include_lines: ['^ERR', '^WARN'] 
 
#============================= Filebeat modules ===============================

filebeat.config.modules:
- module: iis

  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml
  #path: C:\Beats\FileBeat\modules.d\*.yml

  # Set to true to enable config reloading
  reload.enabled: true

  # Period on which files under path should be checked for changes
  reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false


#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  host: "10.1.0.248:5601"

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["10.1.0.248:9200"]

Below is my iis.yml located in .\modules.d

- module: iis
  # Access logs
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

  # Error logs
  error:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

Below is a screenshot of the log location in Windows Server 2012 R2

log%20location

2

Have you tried configuring the log path in iis.yml

3

Filebeat modules can be confusing to configure, because there are several ways to do it. https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-modules.html

If you want to read from IIS using the modules, do not enable filebeat.inputs at all. There are several methods to use modules.

Method #1

This is what is closest to your current solution. Set var.paths in modules.d/iis.yml to your path, so the module know where the input files are located. Then enable the module using ./filebeat modules enable iis. Afterwards you can start Filebeat.

Method #2

I usually use this method during debugging and module development, because in this case no configuration file needs to be edited. You only pass the options via command line.

filebeat -modules=iis -M "iis.access.var.paths=[/path/to/your.log]"

Method #3

You can to enable the IIS module in filebeat.yml. The advantage is that you only need to edit one file which contains all the options. In simple use cases this is sufficient.

filebeat.modules:
- module: iis
  access:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    var.paths:
     - /path/to/your.log
4

Is that section of code that you showed in Method 3 (filebeat.modules:) the same as filebeat.config.modules: ? My yaml file is showing the latter

5

No, it's different. filebeat.config.modules is for options regarding reloading of modules.

You can see filebeat.modules and all options in filebeat.reference.yml: https://github.com/elastic/beats/blob/master/filebeat/filebeat.reference.yml#L12

6

Thanks for clearing that up KCVH. Another question. As I had already ran the command to enable the iis.yml file, if I wanted to just stick with the configuration in my filebeat.yml file, do I have to go back and disable the iis.yml ? Or is having the config in both places, with the iis.yml file being enabled, OK?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.