Cannot see logs in Elasticsearch configured through Filebeat

Elastic Stack Beats
1

Hello,
I have installed Elasticsearch, Kibana and Filebeat but not able to see the logs.

These are filebeat logs

2019-01-23T10:07:49.740+1100	INFO	instance/beat.go:592	Home path: [C:\Users\hbootwala\Downloads\filebeat-6.5.4-windows-x86_64\filebeat-6.5.4-windows-x86_64] Config path: [C:\Users\hbootwala\Downloads\filebeat-6.5.4-windows-x86_64\filebeat-6.5.4-windows-x86_64] Data path: [C:\Users\hbootwala\Downloads\filebeat-6.5.4-windows-x86_64\filebeat-6.5.4-windows-x86_64\data] Logs path: [C:\Users\hbootwala\Downloads\filebeat-6.5.4-windows-x86_64\filebeat-6.5.4-windows-x86_64\logs]
2019-01-23T10:07:49.745+1100	INFO	instance/beat.go:599	Beat UUID: 6349107a-c92b-4ed2-ac23-d7f82878264d
2019-01-23T10:07:49.745+1100	INFO	[beat]	instance/beat.go:825	Beat info	{"system_info": {"beat": {"path": {"config": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64", "data": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64\\data", "home": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64", "logs": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64\\logs"}, "type": "filebeat", "uuid": "6349107a-c92b-4ed2-ac23-d7f82878264d"}}}
2019-01-23T10:07:49.746+1100	INFO	[beat]	instance/beat.go:834	Build info	{"system_info": {"build": {"commit": "bd8922f1c7e93d12b07e0b3f7d349e17107f7826", "libbeat": "6.5.4", "time": "2018-12-17T20:21:46.000Z", "version": "6.5.4"}}}
2019-01-23T10:07:49.746+1100	INFO	[beat]	instance/beat.go:837	Go runtime info	{"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":8,"version":"go1.10.6"}}}
2019-01-23T10:07:49.762+1100	INFO	[beat]	instance/beat.go:841	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-01-21T09:45:28.66+11:00","name":"HBOOTWALA","ip":["fe80::e14b:dd67:8c14:f70a/64","169.254.247.10/16","fe80::6de6:bf4c:1f2a:7ad8/64","172.16.1.105/20","::1/128","127.0.0.1/8"],"kernel_version":"10.0.17134.523 (WinBuild.160101.0800)","mac":["00:09:0f:fe:00:01","34:e6:d7:00:89:60"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"17134.523"},"timezone":"AEDT","timezone_offset_sec":39600,"id":"621a7951-5897-4351-a3ca-bf854449d8e2"}}}
2019-01-23T10:07:49.765+1100	INFO	[beat]	instance/beat.go:870	Process info	{"system_info": {"process": {"cwd": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64", "exe": "C:\\Users\\hbootwala\\Downloads\\filebeat-6.5.4-windows-x86_64\\filebeat-6.5.4-windows-x86_64\\filebeat.exe", "name": "filebeat.exe", "pid": 4284, "ppid": 34400, "start_time": "2019-01-23T10:07:49.635+1100"}}}
2019-01-23T10:07:49.765+1100	INFO	instance/beat.go:278	Setup Beat: filebeat; Version: 6.5.4
2019-01-23T10:07:52.771+1100	INFO	add_cloud_metadata/add_cloud_metadata.go:319	add_cloud_metadata: hosting provider type not detected.
2019-01-23T10:07:52.772+1100	INFO	elasticsearch/client.go:163	Elasticsearch url: http://localhost:9200
2019-01-23T10:07:52.773+1100	INFO	[publisher]	pipeline/module.go:110	Beat name: HBOOTWALA
2019-01-23T10:07:52.781+1100	INFO	elasticsearch/client.go:163	Elasticsearch url: http://localhost:9200
2019-01-23T10:07:52.861+1100	INFO	elasticsearch/client.go:713	Connected to Elasticsearch version 6.5.4
2019-01-23T10:07:52.862+1100	INFO	kibana/client.go:118	Kibana url: http://localhost:5601
2019-01-23T10:08:41.976+1100	INFO	instance/beat.go:717	Kibana dashboards successfully loaded.

This is filebeat.yml

#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: false

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    #- /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*
    - 'C:\Test\*.log'
#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "localhost:5601"

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

Any suggestions?
Thank You.

2

Sometimes, filebeat service does not get started via Windows Service panel.
But when its running, it gives the same logs as posted above.

3

Please format logs and config file using the </> button in the editor.

I don't see any configured input. I assume you want to read from 'C:\Test\*.log'. Setting enabled: true should enable the input.

5

Can see the logs now.
Thanx.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.