Can't setup Winlogbeat or Filebeat dashboards for Kibana

[JulienL]

Hello!

Firstly, sorry for any mistake in my english. I'm not a native english speaker

Hope can someone help me with this.

I have installed and configured ELK with "Basic Security plus HTTPS", I receive logs from multiples server authenticated with API keys. So it's working well here.

The only thing that I can't understand is why I can't setup dashboard with Filebeat or Winlogbeat?

When I run the command sudo filebeat setup --dashboards on Debian or .\winlogbeat.exe setup --dashboards on Windows, this end with the following error:

Loading dashboards (Kibana must be running and reachable)
Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to https://kibana:5601/api/status fails: <nil>. Response: {"statusCode":401,"error":"Unauthorized","message":"Unauthorized"}.

I know it's not recommended in production environnement but the API Keys are using the "elastic" user for testing purposes and to avoid permissions problems.
I'll change this once the problem is solved.

FYI, the CA certificate is installed on the clients and the server.

Did I miss something?

Please find all the configuration files below.

Filebeat /etc/filebeat/filebeat.yml:

filebeat.inputs:
- type: log
  enabled: false
  paths:
    - /var/log/*.log

- type: filestream
  enabled: false
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1


setup.kibana:
  host: "https://kibana:5601"


output.elasticsearch:
  hosts: ["https://kibana:9200"]
  protocol: "https"
  api_key: 'api:key'


setup.ilm:
  enabled: auto
  rollover_alias: "filebeat-srvlinux"
  pattern: "{now/d}-000001"


processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Kibana /etc/kibana/kibana.yml:

server.host: "0.0.0.0"
server.name: "kibana.domain.lan"

elasticsearch.hosts: ["https://kibana.domain.lan:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/DOMAIN-CA.crt" ]

server.ssl.certificate: /etc/kibana/certs/srv-log.crt
server.ssl.key: /etc/kibana/certs/srv-log.key
server.ssl.enabled: true

xpack.encryptedSavedObjects.encryptionKey: censored
xpack.reporting.encryptionKey: censored
xpack.security.encryptionKey: censored

Elasticsearch /etc/elasticsearch/elasticsearch.yml:

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
cluster.name: domain_elk
node.name: SRV-LOG

network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["127.0.0.1"]
discovery.type: single-node

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/certs/elastic-certificates.p12

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/http-SRV-LOG.crt
xpack.security.http.ssl.key: /etc/elasticsearch/certs/http-SRV-LOG.key
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/DOMAIN-CA.crt" ]

Thanks for any kind of help!

[andrewkroh]

I think you are hitting this bug. Kibana authentication is not inheriting the Elasticsearch output api_key value · Issue #24015 · elastic/beats · GitHub

[JulienL]

Hi! Thanks for your answer.

As you suggested, I've set the header manually in the filebeat.yml, and it worked

This look like this now:

setup.kibana:
  host: "https://kibana:5601"
  headers:
    Authorization: "ApiKey censored_base64_key"

Thanks for your help!

[JulienL]

Hello,

I'm back with some problems.
The workaround works well on Windows Systems with Winlogbeat but not on Linux with Filebeat.
Still the same error.

What could be missing?

[andrewkroh]

Are you getting the same 401 error? If so then can you show your config. Maybe it's an indentation issue with the options. The code for Windows and Linux is the same so it should behave the same.