Connecting Filebeat from local machine to existing logstash pipeline

Elastic Stack Beats
1

I'm attempting to push logs from my local machine using Filebeats through an existing logstash collector node to an existing elastic index. Here is my filebeat.yml file:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\Users\ElamR\Documents\ELK\*.log
  exclude_lines: ['^10\.55\.191\..*"GET \/status.*"LTM-Monitor" "-"$']
  fields_under_root: true
  fields:
    secret: "**************"
output.logstash:
  hosts: ["log-evt-dms.els-gp.icp.infineon.com:9889"]

When I run

.\filebeat.exe setup -e

I get the following error message:

{"log.level":"info","@timestamp":"2023-04-25T14:22:50.821-0400","log.origin":{"file.name":"instance/beat.go","file.line":724},"message":"Home path: [C:\\Program Files\\Filebeat] Config path: [C:\\Program Files\\Filebeat] Data path: [C:\\Program Files\\Filebeat\\data] Logs path: [C:\\Program Files\\Filebeat\\logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.824-0400","log.origin":{"file.name":"instance/beat.go","file.line":732},"message":"Beat ID: 8898858b-16d4-44c5-80db-fd1668d81ca3","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.825-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1096},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"C:\\Program Files\\Filebeat","data":"C:\\Program Files\\Filebeat\\data","home":"C:\\Program Files\\Filebeat","logs":"C:\\Program Files\\Filebeat\\logs"},"type":"filebeat","uuid":"8898858b-16d4-44c5-80db-fd1668d81ca3"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.825-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1105},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"a8dbc6c06381f4fe33a5dc23906d63c04c9e2444","libbeat":"8.7.0","time":"2023-03-23T00:44:06.000Z","version":"8.7.0"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.825-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"windows","arch":"amd64","max_procs":8,"version":"go1.19.7"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.843-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1114},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-04-21T08:57:34-04:00","name":"ISCN5CG2236JNF","ip":["fe80::5286:1845:cf6e:ea26","fe80::f3b5:e382:39c8:be9b","fe80::99ab:7d7c:1fae:4af6","10.53.113.81","fe80::55d9:3e7:c40f:1690","::1","127.0.0.1"],"kernel_version":"10.0.19041.2846 (WinBuild.160101.0800)","mac":["5c:60:ba:38:8d:08","70:a8:d3:c8:29:19","70:a8:d3:c8:29:18","70:a8:d3:c8:29:1c"],"os":{"type":"windows","family":"windows","platform":"windows","name":"Windows 10 Enterprise","version":"10.0","major":10,"minor":0,"patch":0,"build":"19045.2846"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"ae50890d-1487-44df-ba9c-321208203cba"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.843-0400","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1143},"message":"Process info","service.name":"filebeat","system_info":{"process":{"cwd":"C:\\Program Files\\Filebeat","exe":"C:\\Program Files\\Filebeat\\filebeat.exe","name":"filebeat.exe","pid":11884,"ppid":16124,"start_time":"2023-04-25T14:22:50.698-0400"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2023-04-25T14:22:50.844-0400","log.origin":{"file.name":"instance/beat.go","file.line":297},"message":"Setup Beat: filebeat; Version: 8.7.0","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-04-25T14:22:53.244-0400","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: ISCN5CG2236JNF","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-04-25T14:22:53.245-0400","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-04-25T14:22:53.245-0400","log.origin":{"file.name":"beater/filebeat.go","file.line":175},"message":"Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-04-25T14:22:53.246-0400","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: index management requested but the Elasticsearch output is not configured/enabled","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: index management requested but the Elasticsearch output is not configured/enabled

Any help I could get to iron this out would be appreciatied. I am doing this to push data to a currently empty test cluster to enable the testing of a script before pushing it to production.

2

This is for importing dashboards in Kibana, create template and ingest pipelines in ES.

To run FB:
.\filebeat -c c:\somepath\filebeat.yml -e

3

Thanks! Was definitely a noob question but all of the Elastic documentation for installing and running filebeats for the first time is geared towards setting up a new pipeline as well I suppose, makes sense but I didn't realize that initially. I need to read more carefully.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.