# \#libbeat **URL:** https://discuss.elastic.co/tag/libbeat/73.md [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) --- ## [Error building Custom beats](https://discuss.elastic.co/t/error-building-custom-beats/381220)
**Author:** [@Dede\_Pessu](https://discuss.elastic.co/u/Dede_Pessu)\ **Replies:** 0\ **Last updated:** [August 22, 2025, 1:43am UTC](https://discuss.elastic.co/t/error-building-custom-beats/381220 "2025-08-22T01:43:18Z")
Attempting to package my custom beat for distribution but getting errors. WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v3) and no specific platform was … --- ## [Beats Kafka Output: support for SASL OAUTH and MSK IAM Authentication](https://discuss.elastic.co/t/beats-kafka-output-support-for-sasl-oauth-and-msk-iam-authentication/368599)
**Author:** [@Jose\_Ledesma](https://discuss.elastic.co/u/Jose_Ledesma)\ **Replies:** 1\ **Last updated:** [November 7, 2024, 4:33pm UTC](https://discuss.elastic.co/t/beats-kafka-output-support-for-sasl-oauth-and-msk-iam-authentication/368599 "2024-11-07T16:33:24Z")
Hi, We are interested in sending logs with Filebeat to an MSK cluster using IAM authentication. Currently, the Kafka Output does not support SASL OAUTH, which is the underlying mechanism for IAM Authentication (see a Go… --- ## [Parse line error: invalid CRI log format when registry offset points to middle of the log event](https://discuss.elastic.co/t/parse-line-error-invalid-cri-log-format-when-registry-offset-points-to-middle-of-the-log-event/365197)
**Author:** [@sasikiranvaddi](https://discuss.elastic.co/u/sasikiranvaddi)\ **Replies:** 5\ **Last updated:** [September 9, 2024, 8:58am UTC](https://discuss.elastic.co/t/parse-line-error-invalid-cri-log-format-when-registry-offset-points-to-middle-of-the-log-event/365197 "2024-09-09T08:58:51Z")
Filebeat: Version: 8.12.1 Autodiscover enabled. We observe at times filebeat reports an error {"log.level":"error","@timestamp":"2024-08-13T10:25:42.693Z","log.logger":"reader\_docker\_json","log.origin":{"function":"g… --- ## [\[bug\] Syslog RFC 5424 parser does not properly handle escaped characters in structured data](https://discuss.elastic.co/t/bug-syslog-rfc-5424-parser-does-not-properly-handle-escaped-characters-in-structured-data/364483)
**Author:** [@pcollardez](https://discuss.elastic.co/u/pcollardez)\ **Replies:** 4\ **Last updated:** [August 7, 2024, 2:16pm UTC](https://discuss.elastic.co/t/bug-syslog-rfc-5424-parser-does-not-properly-handle-escaped-characters-in-structured-data/364483 "2024-08-07T14:16:16Z")
Hello, I'm trying to collect logs from different appliances by using an Elastic Agent 8.14.3 with the Custom UDP Logs integration 1.19.1. One of the appliance is sending RFC 5424 formatted logs, with a structured data … --- ## [Sending data to ES via Google Load balancer a good idea?](https://discuss.elastic.co/t/sending-data-to-es-via-google-load-balancer-a-good-idea/358807)
**Author:** [@buzzdeee](https://discuss.elastic.co/u/buzzdeee)\ **Replies:** 0\ **Last updated:** [May 6, 2024, 10:24am UTC](https://discuss.elastic.co/t/sending-data-to-es-via-google-load-balancer-a-good-idea/358807 "2024-05-06T10:24:41Z")
Hi, I'm want to replace my old cluster in AWS with a new one in GCP, while there, I restructured the layout of the cluster a bit, i.e. have separate master and data nodes, and Kibana running on separate hosts, instead r… --- ## [Module "..." found, but does not contain package "..."](https://discuss.elastic.co/t/module-found-but-does-not-contain-package/356052)
**Author:** [@zeynepyz](https://discuss.elastic.co/u/zeynepyz)\ **Replies:** 1\ **Last updated:** [March 26, 2024, 10:30am UTC](https://discuss.elastic.co/t/module-found-but-does-not-contain-package/356052 "2024-03-26T10:30:52Z")
Hello i am trying to create new metricbeat module for my self improvement. This is my pr. As you can see i get trouble with all at of tests. Some of them are mage check errors. Error: ../../../../go/pkg/mod/github.com/d… --- ## [Documentation on pkg.go.dev (godoc) not working](https://discuss.elastic.co/t/documentation-on-pkg-go-dev-godoc-not-working/344783)
**Author:** [@tlinker13](https://discuss.elastic.co/u/tlinker13)\ **Replies:** 0\ **Last updated:** [October 11, 2023, 5:38am UTC](https://discuss.elastic.co/t/documentation-on-pkg-go-dev-godoc-not-working/344783 "2023-10-11T05:38:46Z")
Hey all, I try to write a first metricbeat module/metricset and need to dive into the MapStrAPI, but there is no content displayed at libbeat's godoc page saying: "Documentation not displayed due to license restriction… --- ## [Measure CPU/Memory for custom libbeat application](https://discuss.elastic.co/t/measure-cpu-memory-for-custom-libbeat-application/343538)
**Author:** [@Nikhil\_Khurana](https://discuss.elastic.co/u/Nikhil_Khurana)\ **Replies:** 0\ **Last updated:** [September 21, 2023, 12:23pm UTC](https://discuss.elastic.co/t/measure-cpu-memory-for-custom-libbeat-application/343538 "2023-09-21T12:23:02Z")
I have built a custom beat using libbeat library that parses my application logs as intended. Is there an out of the box configuration that can be added in yml file to log cpu and memory stats for this beat. I can use Me… --- ## [Fix to libbeats to split bulk requests which are too large, not working in Elastic Agent 8.9.0?](https://discuss.elastic.co/t/fix-to-libbeats-to-split-bulk-requests-which-are-too-large-not-working-in-elastic-agent-8-9-0/342136)
**Author:** [@Craig\_Rodrigues](https://discuss.elastic.co/u/Craig_Rodrigues)\ **Replies:** 6\ **Last updated:** [September 8, 2023, 8:45pm UTC](https://discuss.elastic.co/t/fix-to-libbeats-to-split-bulk-requests-which-are-too-large-not-working-in-elastic-agent-8-9-0/342136 "2023-09-08T20:45:22Z")
In Beats, I see that this commit was merged in March 2023: " Split large batches on error instead of dropping them" PR 34911 I think that PR 34911 changed the Publish() logic: If I look here: func (client \*Client) … --- ## [Error during build for Beats version 8.9.1](https://discuss.elastic.co/t/error-during-build-for-beats-version-8-9-1/341778)
**Author:** [@Nikhitha\_Karennagari](https://discuss.elastic.co/u/Nikhitha_Karennagari)\ **Replies:** 2\ **Last updated:** [August 28, 2023, 12:25pm UTC](https://discuss.elastic.co/t/error-during-build-for-beats-version-8-9-1/341778 "2023-08-28T12:25:28Z")
Hi, I am facing the below error while building the beats repo. Please help to resolve the issue. Thanks Error: running "go build -o build/golang-crossbuild/filebeat-linux-amd64 -buildmode pie -trimpath -tags=withjourna… --- ## [Access container logs with libbeat / filebeat with non-root user](https://discuss.elastic.co/t/access-container-logs-with-libbeat-filebeat-with-non-root-user/340050)
**Author:** [@Nikhil\_Khurana](https://discuss.elastic.co/u/Nikhil_Khurana)\ **Replies:** 0\ **Last updated:** [August 3, 2023, 1:39pm UTC](https://discuss.elastic.co/t/access-container-logs-with-libbeat-filebeat-with-non-root-user/340050 "2023-08-03T13:39:31Z")
I have filebeat / libbeat running as non-root user and want to read docker container logs from /var/lib/docker/container . I have mounted the directory within beat pod but the directory has 700 permission by default, i.e… --- ## [Add\_docker\_metadata cannot process containers that already exited](https://discuss.elastic.co/t/add-docker-metadata-cannot-process-containers-that-already-exited/335435)
**Author:** [@Maciej\_Piasecki](https://discuss.elastic.co/u/Maciej_Piasecki)\ **Replies:** 0\ **Last updated:** [June 7, 2023, 11:46am UTC](https://discuss.elastic.co/t/add-docker-metadata-cannot-process-containers-that-already-exited/335435 "2023-06-07T11:46:29Z")
For the input type container if the log file is discovered after the container is stopped, the add\_metadata\_processor reports {"file.name":"add\_docker\_metadata/add\_docker\_metadata.go","file.line":213},"message":"Contain… --- ## [Ndjson parser doesn't expand keys if target is set](https://discuss.elastic.co/t/ndjson-parser-doesnt-expand-keys-if-target-is-set/334799)
**Author:** [@anon68795679](https://discuss.elastic.co/u/anon68795679)\ **Replies:** 2\ **Last updated:** [May 31, 2023, 1:13pm UTC](https://discuss.elastic.co/t/ndjson-parser-doesnt-expand-keys-if-target-is-set/334799 "2023-05-31T13:13:05Z")
Hi, it seems that there is the same issue with the ndjson parser like in the decode\_json\_fields processor some time ago: Expand fields in \`decode\_json\_fields\` if target is set by kvch · Pull Request #32010 · elastic/bea… --- ## [Filebeat's add\_docker\_metadata stop working after container restart](https://discuss.elastic.co/t/filebeats-add-docker-metadata-stop-working-after-container-restart/322358)
**Author:** [@kowy](https://discuss.elastic.co/u/kowy)\ **Replies:** 1\ **Last updated:** [January 16, 2023, 12:22pm UTC](https://discuss.elastic.co/t/filebeats-add-docker-metadata-stop-working-after-container-restart/322358 "2023-01-16T12:22:00Z")
We use filebeat from Graylog to collect logs from services run in docker-compose. Filebeat is not started as a linux service, but executed as this command: sudo /usr/share/filebeat/bin/filebeat --path.home /usr/share/f… --- ## [Filebeat hogged the IO](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581)
**Author:** [@silence-linhl](https://discuss.elastic.co/u/silence-linhl)\ **Replies:** 0\ **Last updated:** [September 16, 2022, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-hogged-the-io/314581 "2022-09-16T14:35:20Z")
filebeat hogged the IO After starting filebeat, I found that the IO of the machine became very high. After the digging, it was found that filebeat kept writing to disk a file named checkpoint.new after it was started. S… --- ## [Type usage in libbeat/outputs/elasticsearch/client causes issue on OpenSearch 2.0+ engine](https://discuss.elastic.co/t/type-usage-in-libbeat-outputs-elasticsearch-client-causes-issue-on-opensearch-2-0-engine/306166)
**Author:** [@Suraj\_Singh](https://discuss.elastic.co/u/Suraj_Singh)\ **Replies:** 2\ **Last updated:** [June 1, 2022, 9:40pm UTC](https://discuss.elastic.co/t/type-usage-in-libbeat-outputs-elasticsearch-client-causes-issue-on-opensearch-2-0-engine/306166 "2022-06-01T21:40:44Z")
With \_type removal in OpenSearch 2.0+ (previously deprecated in Elasticsearch 7.x); beat still uses \_type as DocType while building BulkRequest meta data; resulting in illegal\_argument\_exception from OpenSearch engine. E… --- ## [Upgrade to Version 8](https://discuss.elastic.co/t/upgrade-to-version-8/304052)
**Author:** [@norgro2601](https://discuss.elastic.co/u/norgro2601)\ **Replies:** 1\ **Last updated:** [May 5, 2022, 10:47pm UTC](https://discuss.elastic.co/t/upgrade-to-version-8/304052 "2022-05-05T22:47:59Z")
Hello, I want to upgrade the Elastic Stack to Version 8. Because we have some AIX Servers in our environment, I used the Beats that were made available by Bull Freeware some years ago and are running version 7.5.x. As … --- ## [Filebeat kafka input using multiline parser gives no output](https://discuss.elastic.co/t/filebeat-kafka-input-using-multiline-parser-gives-no-output/292290)
**Author:** [@rsniper](https://discuss.elastic.co/u/rsniper)\ **Replies:** 1\ **Last updated:** [December 23, 2021, 5:27am UTC](https://discuss.elastic.co/t/filebeat-kafka-input-using-multiline-parser-gives-no-output/292290 "2021-12-23T05:27:39Z")
I have posted the issue on stackoverflow as well Filebeat kafka input using multiline parser gives no output Filebeat is configured to use input from kafka and output to file When the multiline setting is turned off, … --- ## [Libbeat or Official Beat Delivery Guarantees with Logstash Ingest?](https://discuss.elastic.co/t/libbeat-or-official-beat-delivery-guarantees-with-logstash-ingest/292220)
**Author:** [@benatsb](https://discuss.elastic.co/u/benatsb)\ **Replies:** 0\ **Last updated:** [December 16, 2021, 8:35pm UTC](https://discuss.elastic.co/t/libbeat-or-official-beat-delivery-guarantees-with-logstash-ingest/292220 "2021-12-16T20:35:42Z")
Hello, I'm looking for an overview and in-depth details on elastic beat delivery guarantees including queuing with Logstash, but information is hard to find... I have only been able to find information regarding Filebe… --- ## [Test Suite error - time.Local vs time.UTC?](https://discuss.elastic.co/t/test-suite-error-time-local-vs-time-utc/286735)
**Author:** [@hinchliff](https://discuss.elastic.co/u/hinchliff)\ **Replies:** 1\ **Last updated:** [October 15, 2021, 2:04pm UTC](https://discuss.elastic.co/t/test-suite-error-time-local-vs-time-utc/286735 "2021-10-15T14:04:11Z")
Seeing some failures for make testsuite for libbeat, that seem to be Timezone related. (My computer is in US Eastern.) command \[go test -cover -coverprofile /tmp/gotestcover-1560318489 github.com/elastic/beats/v7/libbe… --- ## [\[bug query\] duplication mac address in hosts metadata - concern?](https://discuss.elastic.co/t/bug-query-duplication-mac-address-in-hosts-metadata-concern/278370)
**Author:** [@kortschak](https://discuss.elastic.co/u/kortschak)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 12:51pm UTC](https://discuss.elastic.co/t/bug-query-duplication-mac-address-in-hosts-metadata-concern/278370 "2021-07-13T12:51:00Z")
Hi, While working through the creating a beat tutorial I noticed that mac addresses may be duplicated in the hosts metadata on linux. This occurs for virbr interfaces due to virtual bridges generating two interfaces, sh… --- ## [V7.13.0 breaks OSS distributions](https://discuss.elastic.co/t/v7-13-0-breaks-oss-distributions/274568)
**Author:** [@Dan\_Bason](https://discuss.elastic.co/u/Dan_Bason)\ **Replies:** 3\ **Last updated:** [June 1, 2021, 7:26pm UTC](https://discuss.elastic.co/t/v7-13-0-breaks-oss-distributions/274568 "2021-06-01T19:26:40Z")
I've only tested with Metricbeat, but I assume most of the beats will be affected in v7.13.0. Connections to OSS distributions of Elasticsearch now fail due to a new license check introduced in libbeat (they throw an er… --- ## [Negative Kafka partition bug](https://discuss.elastic.co/t/negative-kafka-partition-bug/270046)
**Author:** [@cbrown184](https://discuss.elastic.co/u/cbrown184)\ **Replies:** 1\ **Last updated:** [April 13, 2021, 11:38pm UTC](https://discuss.elastic.co/t/negative-kafka-partition-bug/270046 "2021-04-13T23:38:01Z")
Hi - we use Filebeat to output our logs to Kafka. We got hit by a nasty bug in prod where Filebeat gets stuck in an endless loop. It was previously documented on this thread Filebeat kafka output hash.hash get negative … --- ## [Filebeat kafka output hash.hash get negative partition cause stuck](https://discuss.elastic.co/t/filebeat-kafka-output-hash-hash-get-negative-partition-cause-stuck/265153)
**Author:** [@Vvv](https://discuss.elastic.co/u/Vvv)\ **Replies:** 0\ **Last updated:** [February 23, 2021, 5:27am UTC](https://discuss.elastic.co/t/filebeat-kafka-output-hash-hash-get-negative-partition-cause-stuck/265153 "2021-02-23T05:27:58Z")
Summarry I use filebeat to collect logs and output to kafka. Due to partition.hash.hash config hash message then mod to select partition, in some situation, this may cause filebeat stuck, e.g.: massage: 2304669687 hash… --- ## [JWT Support in Beats](https://discuss.elastic.co/t/jwt-support-in-beats/254825)
**Author:** [@cchandak](https://discuss.elastic.co/u/cchandak)\ **Replies:** 2\ **Last updated:** [November 12, 2020, 9:38pm UTC](https://discuss.elastic.co/t/jwt-support-in-beats/254825 "2020-11-12T21:38:32Z")
I would like to scrape logs using Beats (either Filebeat, Metricbeat, etc.) and send them to Logstash or Kafka depending on my use case. Currently, I am leveraging the SSL support, but I would like to use JSON Web Token … --- ## [Custom Unpack for configuration](https://discuss.elastic.co/t/custom-unpack-for-configuration/254306)
**Author:** [@tomkirk](https://discuss.elastic.co/u/tomkirk)\ **Replies:** 5\ **Last updated:** [November 5, 2020, 11:15am UTC](https://discuss.elastic.co/t/custom-unpack-for-configuration/254306 "2020-11-05T11:15:05Z")
I'm working on a beat where I have currently in my configuration an array of string like this: fields: - FieldName - AnotherFieldName What I would like is the ability to optionally provide a type tag, like so: fie… --- ## [\[feature request\] Add URL Parse processors](https://discuss.elastic.co/t/feature-request-add-url-parse-processors/253084)
**Author:** [@OhBonsai](https://discuss.elastic.co/u/OhBonsai)\ **Replies:** 0\ **Last updated:** [October 23, 2020, 5:21am UTC](https://discuss.elastic.co/t/feature-request-add-url-parse-processors/253084 "2020-10-23T05:21:04Z")
Hi All: It's very common url format field in log. urldecode processor have been implements in libbeat. Add a net/url.Parse processor will be helpful :slight\_smile: I have already done this feature? May I open a Pull… --- ## [Latest released version (v7.9.1) python environment fails](https://discuss.elastic.co/t/latest-released-version-v7-9-1-python-environment-fails/249568)
**Author:** [@Sean\_Houghton](https://discuss.elastic.co/u/Sean_Houghton)\ **Replies:** 1\ **Last updated:** [September 22, 2020, 6:00pm UTC](https://discuss.elastic.co/t/latest-released-version-v7-9-1-python-environment-fails/249568 "2020-09-22T18:00:47Z")
When using the latest pinned version (as everyone should be doing in production) the python-dev makefile target fails with setuptools errors ImportError: cannot import name 'Feature' from 'setuptools' (/private/tmp/… --- ## [Make Inside libbeat fails](https://discuss.elastic.co/t/make-inside-libbeat-fails/248602)
**Author:** [@Ali\_Tahir](https://discuss.elastic.co/u/Ali_Tahir)\ **Replies:** 4\ **Last updated:** [September 15, 2020, 8:42am UTC](https://discuss.elastic.co/t/make-inside-libbeat-fails/248602 "2020-09-15T08:42:17Z")
I get the following error on running make inside libbeat go build -ldflags "-X github.com/elastic/beats/libbeat/version.buildTime=2020-09-15T05:09:36Z -X github.com/elastic/beats/libbeat/version.commit=3435fe9f46e53708d…