# Duplicate messages created by Journalbeat 6.7.1-1 **URL:** \ **Category:** Beats\ **Tags:** journalbeat\ **Created:** [April 8, 2019, 11:35pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930 "2019-04-08T23:35:30Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![ddiguru](https://avatars.discourse-cdn.com/v4/letter/d/7c8e57/32.png) [@ddiguru](https://discuss.elastic.co/u/ddiguru)\ **Post date:** [April 8, 2019, 11:35pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/1 "2019-04-08T23:35:30Z")
I'm running journalbeat-6.7.1-1 on CentOS 7.6 and Fedora 28, and i'm noticing that journal beat is duplicating _some_ but NOT all messages. The only thing that appears different between the messages is the journalbeat\_read\_timestamp field. Exampe: Journalctl output shows four (4) messages: ``` Apr 08 17:20:05 fedora.ddiguru.net systemd[1]: Starting system activity accounting tool... Apr 08 17:20:05 fedora.ddiguru.net audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=sysstat-collect comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Apr 08 17:20:05 fedora.ddiguru.net audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=sysstat-collect comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Apr 08 17:20:05 fedora.ddiguru.net systemd[1]: Started system activity accounting tool. ``` Journalbeat sent the following five (5) messages to my Graylog/Elasticsearch cluster(s): ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/2/52befe750c2719882aa2ab80a6ca49f2ddc151ab.png) My configuration is fairly simple: ``` #----------------------------------------- Journalbeat Input -------------------------------------------------- journalbeat.inputs: - paths: - "/var/log/journal/9926d0e1c68d4a1e9c9a69fd51bd65d8/system.journal" seek: cursor cursor_seek_fallback: head #------------------------------------------- Logstash Output ------------------------------------------------- output.logstash: hosts: ["192.168.4.37:5044"] seccomp.enabled: false ``` Is there something i'm missing? Do i need to tune this with a memory queue? other? Please advise. Thanks ---
**Author:** ![John\_Lemsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/john_lemsky/32/43286_2.png) [@John\_Lemsky](https://discuss.elastic.co/u/John_Lemsky)\ **Post date:** [April 9, 2019, 7:16am UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/2 "2019-04-09T07:16:04Z")
I probably have the same problem. Can you confirm or deny what these duplicate messages appear after journald rotates log? ---
**Author:** ![ddiguru](https://avatars.discourse-cdn.com/v4/letter/d/7c8e57/32.png) [@ddiguru](https://discuss.elastic.co/u/ddiguru)\ **Post date:** [April 9, 2019, 1:11pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/3 "2019-04-09T13:11:12Z")
This morning it appeared that logging had stopped as a result of the log rollover. I have a cron job that runs hourly to restart the service that looks something like this: ``` #!/bin/sh set -ue SYSTEM_JOURNAL_FILE_ID=$(journalctl --file /run/log/journal/71da943ecfb04e7a8995f336453a9bf3/system.journal --header | awk -F ': ' '/File ID/ { print $2 }') if [! -e /var/lib/journalbeat/system-journal.id] || ["$SYSTEM_JOURNAL_FILE_ID" != "$(cat /var/lib/journalbeat/system-journal.id)"]; then systemctl try-restart journalbeat && echo "$SYSTEM_JOURNAL_FILE_ID" > /var/lib/journalbeat/system-journal.id fi ``` This was a script i found while troubleshooting the systemd.log rollover issue/bug. After restarting promptly at 0700, yes, i'm still getting duplicated messages. ---
**Author:** ![ddiguru](https://avatars.discourse-cdn.com/v4/letter/d/7c8e57/32.png) [@ddiguru](https://discuss.elastic.co/u/ddiguru)\ **Post date:** [April 9, 2019, 1:18pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/4 "2019-04-09T13:18:05Z")
I've looked back in time and can ascertain that this issue started in 6.7.0, I viewed all messages that came in from 6.6.2 journalbeats and this problem did not exist. ---
**Author:** ![rwaffen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rwaffen/32/43791_2.png) [@rwaffen](https://discuss.elastic.co/u/rwaffen)\ **Post date:** [April 9, 2019, 3:09pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/5 "2019-04-09T15:09:24Z")
can conform this. 6.7.0 and 6.7.1 send duplicates. with 6.6.2 everything is fine. ---
**Author:** ![ikeastaff](https://avatars.discourse-cdn.com/v4/letter/i/9fc29f/32.png) [@ikeastaff](https://discuss.elastic.co/u/ikeastaff)\ **Post date:** [April 10, 2019, 2:35pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/6 "2019-04-10T14:35:29Z")
I am seeing the same issues. I think sometimes with the same read\_timestamp as well but mostly this is the only field (besides the id) where the records differ. ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/5/95d768960cd6ed7e894954c0e9eebd37fcf39611.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/b/cb431fda7f39eff269a2f965a933114e3a77ead9.png) ---
**Author:** ![ikeastaff](https://avatars.discourse-cdn.com/v4/letter/i/9fc29f/32.png) [@ikeastaff](https://discuss.elastic.co/u/ikeastaff)\ **Post date:** [April 10, 2019, 7:45pm UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/7 "2019-04-10T19:45:42Z")
Same problem in the just released 7.0.0 stack. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 7:02am UTC](https://discuss.elastic.co/t/duplicate-messages-created-by-journalbeat-6-7-1-1/175930/8 "2022-11-04T07:02:50Z")