Logs from DataFlow is streaming but not showing in Kibana

Elastic Stack Elasticsearch
1

Certainly! Here’s an improved version of your message:

Hello,

I am currently using the provided documentation to create a Dataflow job that pulls our application logs into Pub/Sub. Here is the process I followed:

  1. Created a Pub/Sub topic for our application logs.
  2. Subscribed to the topic and used it as the dataset for Dataflow.
  3. Configured the GCP integration with Pub/Sub, naming the dataset "pubsub".
  4. Deployed the Dataflow job, which is running successfully in GCP.

However, I am unable to locate the logs in Kibana. I have checked both the "Discover" and "Observability" sections, but no logs appear.

Please note that I am not looking to create a dashboard; I simply want to document the search format in Kibana. Could you advise if this is the correct approach, or should I be using the Elastic Agent for this use case?

Thank you.