Filebeat yml config issue

The index for my Filebeat YAML configuration is not appearing in the Kibana index pattern. This issue arose with a new Filebeat setup on the Linux server.

I configured Filebeat and created a YAML configuration file. I verified the syntax using a YAML checker, and it's correct. Additionally, I used the command "filebeat test config -c /xxx/xxx/xx" to confirm that the configuration format is fine.

I manually created an index with the command "curl -XPUT 'httpxxxxxxx:port/aaaaaa_bbb'" and checked the indices using "curl -XGET 'httpxxxxxx:port/_cat/indices,'" receiving the expected output. However, in Kibana, I couldn't locate the index pattern or set the timestamp. Despite trying various methods, none have been successful. I'm uncertain about the error, and I seek assistance in identifying the issue.

Log files are stored in the .out format, such as "xxxxxxxx_20230419_132134.out."

I appreciate any help in resolving this issue within the Elastic community.

###################### Filebeat Configuration #########################

filebeat.inputs:

  • type: log
    id: my-logs
    enabled: true
    paths:
    • /xxxxx/logs/*.out # Update the file extension to .out

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

setup.template.settings:
index.number_of_shards: 1
output.elasticsearch:
hosts: ["xxxxxxxxxxxx:9200"]
indices:
- index: "aaaaaa_abc01_xyz"
when.contains:
log.file.path: "/xxxxxxxx/logs/"

processors:

  • add_host_metadata:
    when.not.contains.tags: forwarded
  • add_cloud_metadata: ~
  • add_docker_metadata: ~
  • add_kubernetes_metadata: ~

logging.level: debug

After you start filebeat, what shows up in your systemctl logs for filebeat? there should be some hints there if the data is ingested or not. Also you do not need to create the index yourself, filebeat will do it.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.