UDP-input Receiving an encoding value �

Elastic Stack Logstash
1

Hi

I am using logstash udp input and in elasticsearch field event.original have true values. but in a document field.DeviceCapabilities value is "�" and for field.PoleCapabilities is empty.

fieldname: event.original Value: {"fields":{"DeviceCapabilities":"\ufffd\u0004\u0002\u0000\u0000\u0000\u0000\u0000", "PoleCapabilities":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"},"timestamp":1692170702}

field.DeviceCapabilities : �
field.PoleCapabilities:
"timestamp":1692170702 how I can get my true values in field.
THANKS

2

You should add encoder UTF-16 or ISO-8859-1. Full list you have here
Add to input udp
codec => plain { charset=>"UTF-16" }

3

Not Working

4

Does this happen for the Entire message or just a couple of fields in some messages? if it is just a couple of fields in some messages, then this does not seem to be a Logstash issue, your source that is sending the data with a different encoding on those fields.

An encoding issue would affect the entire message.

1 Like
5

its just 2 fields in message
can i use any filter to decode particular fields

6

If you get \ufffd in your field I believe it is already too late. The data decoder could not interpret the character so it was replaced.

7

Thanks !!
I will handle this on the source end.

8

Yes, it would be best. ELK use UTF-8.

By the way, I have used several UTF-16 decoders but couldn't convert the DeviceCapabilities field. Not sure should DeviceCapabilities be a number, string or special characters.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.