How I can send logs from filebeat to Elasticsearch in another GKE

Elastic Observability Logs
1

Hi!

I have two GKE clusters in different GCP projects.

GKE clusters:
1) GKE-ELASTIC (with kibana and elastic)
2) GKE-APPS (with my microservices)

In the GKE-APPs I disabled the gke logging, because I want to redirect all logging to my "external"/dedicated gke-elastic. For that I installed the fluent-bit.

Step:
helm install --name fluentbit stable/fluent-bit

I guest its the only thing I need to install in my GKE-Apps cluster right? but the point is find the right setup for the backend.

backend:
  type: es
  es:
    host: 10.148.0.242
    port: 9200
    index: kubernetes_cluster
    type: flb_type
    logstash_prefix: kubernetes_cluster
    replace_dots: "On"
    time_key: "@timestamp"
    http_user: "elastic"
    http_passwd: "vt8sv2zhrr2nbpkdgh422flc"
    tls: "off"
    tls_verify: "on"
    tls_ca: ""
    tls_debug: 1

I try with the service name, with the clusterIP, but nothing...

The error always is like that

[2019/08/01 15:36:17] [ info] [filter_kube] API server connectivity OK
[2019/08/01 15:36:17] [ info] [sp] stream processor started
[2019/08/01 15:36:18] [error] [http_client] broken connection to 10.148.0.242:9200 ?
[2019/08/01 15:36:18] [ warn] [out_es] http_do=-1 URI=/_bulk
[2019/08/01 15:36:18] [error] [http_client] broken connection to 10.148.0.242:9200 ?
[2019/08/01 15:36:18] [ warn] [out_es] http_do=-1 URI=/_bulk
[2019/08/01 15:36:18] [error] [http_client] broken connection to 10.148.0.242:9200 ?

Do you know what configuration should I try?

Any suggestions will be appreciated!

Thank you

2

Hi @David_Oceans,

I'm not an expert in things related to fluent bit or GKE, but from the ip address I guess that 10.0.0.0/8 addresses are not routed between k8s clusters without extra configuration. It might be worth investigating how the two clusters can be connected, e.g. via a VPC.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.