ELK Stack Setup on GCP

Elastic Stack Elasticsearch
1

Hi Everyone,

We are looking to setup ELK stack on Google Cloud Platform and would need some suggestions to build a high availability cluster.

We have disabled Stackdriver Alerts for Dev/QA Environments and are looking to use ELK as its an Open Source Alternative for Log Monitoring and Analysis. We also have Kubernetes Clusters, through which we get our application logs.

Would you recommend to build a ELK cluster on GKE and run ELK components as Pods or do I set it up as Virtual Machines ?

Also, as there is no filebeat client involved, how do I pull logs from GCP ?
We have created exports ( sinks ) in Google Cloud Platform which can be used as Pub/Sub. But, we are not sure how to setup a connection between GCP Logs and Push/Pull Logs to Logstash/Elasticsearch

2

I found this https://www.elastic.co/guide/en/logstash/current/plugins-inputs-google_pubsub.html , Is this the Only Solution ?

Will it help me get logs from Kubernetes Cluster ( Fluentd) ?

3

One you setup the sink use the Cloudpubsub plugin https://logz.io/blog/google-pub-sub/

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.