# Filebeat disk queue messing up encoding **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 8, 2021, 1:00pm UTC](https://discuss.elastic.co/t/filebeat-disk-queue-messing-up-encoding/260546 "2021-01-08T13:00:25Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Valker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/valker/32/43179_2.png) [@Valker](https://discuss.elastic.co/u/Valker)\ **Post date:** [January 8, 2021, 1:00pm UTC](https://discuss.elastic.co/t/filebeat-disk-queue-messing-up-encoding/260546/1 "2021-01-08T13:00:25Z")
Hello, I believe I have found a misbehavior in the disk queue mechanism in the filebeat. When I’m adding a UTF-8 encoded json with a polish characters, on the output I am getting messed up value. Example: ^^^^^^^^^^^^^^^^^^^^^ IN: "||OUT1|OUT2|\* nie **słu** szne lub b **łę** dne obciążenie ... pod **wój** ne obciążenie ... **całości** lub części środków ...TABELI PONIŻEJ i ZATWIERDŹ.|" OUT: "||OUT1|OUT2|\* nie **su** suszne lub b **ę** dndne obciżenenie ... pod **wj** njne obciżenenie ... **cao\ufffdocici** lub czścici rorodkw w ...TABELI PONIEJEJ i ZATWIERD.|.|" ^^^^^^^^^^^^^^^^^^^^^^ This is happening only when the disk queue mechanism is enabled. When its off everything working as expected. Also the problem is only affecting fields that contain inner encoded json (field “executionAdditionalInfo” in the example below). For fields with just text its working fine (e.g field “modUserFullName”). - Version: **7.10.1** - Operating System: **Windows 10 Enterprise** (but also Windows Server) - Steps to Reproduce: 1. Download filebeat version 7.10.1 (for windows) 2. Create folders named “input” and “output”. 3. Replace filebeat.yml with: [https://gist.github.com/eValker/f5ae0d046d7b6807f2ea5a4cab525dac#file-filebeat-yml](https://gist.github.com/eValker/f5ae0d046d7b6807f2ea5a4cab525dac#file-filebeat-yml) 4. Start filebeat. 5. Create file input/test.log with content: [https://gist.github.com/eValker/f5ae0d046d7b6807f2ea5a4cab525dac#file-test-log](https://gist.github.com/eValker/f5ae0d046d7b6807f2ea5a4cab525dac#file-test-log) (remember to leave empty line at the end of the file) 6. Check the output file: “_output/filebeat\_out_”. Everything is working as expected (both "_modUserFullName_" and "_executionAdditionalInfo_" are ok - same as inout). 7. In the filebeat.yml uncomment lines **35-38** (enable queue.disk). Then restart the filebeat service. 8. Copy down the first line of the input file (so the service will re-read it). Remember about leaving one empty line 🙂 9. Check the output. Characters in the field “executionAdditionalInfo” are messed up. 10 (optional): You can revert step 7 (disable queue.disk again) then restart service again. Everything is working ok again 🙂 I did not found an option to change encoding in the queue disk mechanism in the docs: [https://www.elastic.co/guide/en/beats/filebeat/current/configuring-internal-queue.html#configuration-internal-queue-disk](https://www.elastic.co/guide/en/beats/filebeat/current/configuring-internal-queue.html#configuration-internal-queue-disk) If it is my fault please help 🙂 PS. I am using "@" character at the beginning of the "executionAdditionalInfo" field on the purpose. I do not want this field to be parsed as json. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 5, 2021, 3:00pm UTC](https://discuss.elastic.co/t/filebeat-disk-queue-messing-up-encoding/260546/2 "2021-02-05T15:00:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.