# Filebeat journald input truncates custom fields at ~64KiB

**URL:** <https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 8, 2024, 8:57am UTC](https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833 "2024-02-08T08:57:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrflibble](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mrflibble](https://discuss.elastic.co/u/mrflibble)\
**Post date:** [February 8, 2024, 8:57am UTC](https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833/1 "2024-02-08T08:57:16Z")

</div>

I have an issue where custom fields in systemd-journald entries are being truncated at ~64KiB. The regular "MESSAGE" field doesn't seem to be affected, only custom fields.

I can't seem to find an option in the docs that might allow for longer fields, am I missing something obvious?

To reproduce:

```yaml
# filebeat.yml
filebeat.inputs:

- type: journald
  include_matches.match:
    - SYSLOG_IDENTIFIER=test

output.file:
  path: "/tmp/filebeat"
  filename: filebeat

```

```js
// Create a journal event with a custom field called "data", with 89784 chars of text
const Journald = require('systemd-journald');
const logger = new Journald({syslog_identifier: 'test'});
const data = 'abcdefghijklmnopqrstuvwxyz...'.repeat(3096)
logger.info('Test message', {"data": data}) 

```

Verify that the length is correct in systemd-journald (aside from an added trailing newline)

```bash
journalctl MESSAGE="Test message" -o json --all --no-pager|jq -r .DATA | wc -m
> 89785

```

Look at the filebeat output, see that it has been truncated.

```bash
cat /tmp/filebeat/filebeat-20240207.ndjson | jq -r .journald.custom.data | wc -m
> 65532

```

If I enable debug logging in Filebeat I see the truncated message in a processing/processors.go "Publish event" log message, so I suspect it's an input issue, not an output issue. In production I use a logstash output and have the same problem.

Any suggestions how I might get round this limit?

---

<div class="post-metadata">

**Author:** ![mrflibble](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mrflibble](https://discuss.elastic.co/u/mrflibble)\
**Post date:** [February 29, 2024, 8:09am UTC](https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833/2 "2024-02-29T08:09:08Z")

</div>

Optimistically giving this a bump. I've started digging around the source, but my go-fu is weak and I don't immediately see any obvious issues.

---

<div class="post-metadata">

**Author:** ![mrflibble](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mrflibble](https://discuss.elastic.co/u/mrflibble)\
**Post date:** [March 20, 2024, 9:19am UTC](https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833/4 "2024-03-20T09:19:27Z")

</div>

Hi, is this an AI generated answer? The listing of not-quite-relevant suggestions, ending with a summary, sounds like a lot of the LLM output I've seen recently.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2024, 11:19am UTC](https://discuss.elastic.co/t/filebeat-journald-input-truncates-custom-fields-at-64kib/352833/5 "2024-04-17T11:19:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.