Filebeat ingest

Anwar · February 25, 2022, 5:02am

Hi Team,

I want to use the below ingest directly in the Filebeat.yml file.

elastic/beats/blob/6b116c5b837e1760b787fbe4d02ca9a6088d1beb/x-pack/filebeat/module/aws/cloudtrail/ingest/pipeline.yml

---
description: Pipeline for AWS CloudTrail Logs
processors:
  - rename:
      field: "message"
      target_field: "event.original"
  - json:
      field: "event.original"
      target_field: "json"
  - date:
      field: "json.eventTime"
      target_field: "@timestamp"
      ignore_failure: true
      formats:
        - ISO8601
  - rename:
      field: "json.eventVersion"
      target_field: "aws.cloudtrail.event_version"
      ignore_failure: true
  - rename:

This file has been truncated. show original

I have made changes for few processor. need to know -> how to use grok, Geoip, user-agent and scripts.
Please help me on the same.

Because, am using Filebeat.yml with Cloudtrail module for pushing the Cloudtrail log event to the elk using docker.
The logs in elk is not parsed as expected.
Note: I want to push it to Elasticsearch via Logstash only.

system · March 25, 2022, 7:03am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Moving from ELK to EFK Beats docker , filebeat	3	1203	November 26, 2019
Filebeat custom module pipeline failed Beats filebeat	5	511	October 20, 2020
Filebeat not uploading lastest ingest grok pattern to elasticsearch Beats filebeat	6	742	May 31, 2019
Filebeat 7.10 aws cloudtrail is not parsing the output as shown in documentation Beats beats-module , filebeat	2	340	March 24, 2022
Filebeat not sending logs to elasticsearch when pipeline is configured in filebeat.yml Beats filebeat	2	843	February 20, 2020

Filebeat ingest

Related topics