Ingest already created doc in pubsub to elk using logstash


I am using microservices and part of the job by microservices is to create a doc with index and doc id and other fields that will be pushed to pubsub.

Currently, we are ingesting to elk by pulling messages from GCP pubsub using python es (not bulk api).
This process is too consuming so created a POC to ingest using Logstash, but logstash ingest the messages from pubsub as an event.

Now I want to know if can ingest the original message directly to elk without other fields that logstash adds like timestamp, host, event.message and event.original.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.