Filebeat how to set a custom indice

Elastic Stack Elasticsearch
1

I have never been more frustrated in a product than I am with the ELK stack. Its needlessly complicated, feels almost intentional as if to push people towards Fleet and Elastic Cloud.

I have given up on the Apache integration. Given up on logstash.
I finally have a dedicated linux install showing up in kibana with only the Filebeat service dumping to external output.elasticsearch.

My Apache logformat is pretty common:
LogFormat "%h %l %u %t "%r" %q %>s %b" common

but it is dumping into Elastic as one message.

I tried to enable filebeat setup
to set the index / dash but I get an error that my kibana is localhost despite my config point to Kibana Host by IP and no localhost anywhere in config.

Error creating Kibana client: fail to get the Kibana version: HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status: dial tcp [::1]:5601: connect: connection refused. Response: .
Exiting: fail to create the Kibana loader: Error creating Kibana client: Error creating Kibana client: fail to get the Kibana version: HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status:

filebeat.yml:
setup.kibana:
hosts: ["10.0.0.5:5601"]

I get around this by declaring the host:
filebeat setup -e -E setup.kibana.host=10.0.0.5:5601

Loaded index template
Loaded dashboards
Loaded machine learning job configurations

Still ssl_access_logs lines show only as single message field.

I tried enabling the Apache module but it added nothing.

I have and index called Weblog that has the patterns brought over from another setup.

I want to change the filebeat to point to that index.

Output.elasticsearch:
#Array of hosts to connect to.
hosts: ["10.0.0.5:9200"]
#TRYING TO SET A CUSTOM INDEX
index: ["weblog"]

ERROR instance/beat.go:906 Exiting: can not convert 'object' into 'string' accessing 'output.elasticsearch.index' (source:'/etc/filebeat/filebeat.yml')
Exiting: can not convert 'object' into 'string' accessing 'output.elasticsearch.index' (source:'/etc/filebeat/filebeat.yml')

How do I point filebeat to a set index ?

2

I removed the brackets on index and now I get a new error

Output.elasticsearch:
#Array of hosts to connect to.
hosts: ["10.0.0.5:9200"]
#TRYING TO SET A CUSTOM INDEX
index: weblog

ERROR instance/beat.go:906 Exiting: setup.template.name and setup.template.pattern have to be set if index name is modified
Exiting: setup.template.name and setup.template.pattern have to be set if index name is modified

3

I added the lines for template and it seems to work.
now it goes to my correct index weblog where my patterns are recognized for each of the log columns.

output.elasticsearch:
hosts: [(ip):9200"]
index: weblog

setup.template.name: "elasticissh!t"
setup.template.pattern: "elasticissh!t*"

shame I can't get the apache dashboards from the integration.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.