Received an event that has a different character encoding than you configured

Elastic Stack Logstash
1

I have seen several threads related to the same error, but none of them seem to address the issue as we are experiencing it. The messages I get this error for are from ADAudit+ reading logs from an Isilon storage appliance. The message is plain-text, as you can see in the Logstash error below, but I am wondering if the file path is causing the message to be interpreted as binary? You can see the literal string \\xB0 in the file path, but I have confirmed this is the actual name of the folder on the Isilon.

Feb 09 09:09:22 LS3 logstash[19327]: [2021-02-09T09:09:22,648][WARN ][logstash.codecs.line     ][main][c0f4a18fa7c1a4e966ba29a207a3e1e5fd759cc188013a3bb9c4aa090c7e5323] Received an event that has a different character encoding than you configured. {:text=>"<110>1 2021-02-09T08:24:29.000-08:00 ServerName ADAuditPlus - - -  [ Category = FileAuditReports ]  [ REPORT_PROFILE = File (or) Folder Created ]  [ EVENT_NUMBER = 46560 ]  [ TIME_GENERATED = 1612887869 ]  [ EVENT_TYPE = 8 ]  [ EVENT_TYPE_TEXT = Success ]  [ SOURCE = Isilon.Contoso.com ]  [ REMARKS = An attempt was made to access an object ]  [ HANDLE_ID = null ]  [ OBJECT_NAME = \\\\ifs\\\\ShareName\\\\Arbitrary\\\\File\\\\Path\\\\To\\\\2021\\\\42100030\\\\HSM\\\\0030D302_2\\xB0\\\\t ]  [ UNC_NAME = \\\\\\\\Isilon.Contoso.com\\\\Arbitrary\\\\File\\\\Path\\\\To\\\\2021\\\\42100030\\\\hsm\\\\0030d302_2\\xB0\\\\t ]  [ FILE_NAME = t ]  [ FILE_LOCATION = \\\\ifs\\\\ShareName\\\\Arbitrary\\\\File\\\\Path\\\\To\\\\2021\\\\42100030\\\\HSM\\\\0030D302_2\\xB0\\\\ ]  [ LOGON_ID = null ]  [ DOMAIN = Contoso.com ]  [ ACCESSES = CREATED ]  [ PROCESS_ID = null ]  [ PROCESS_NAME = null ]  [ CLIENT_HOST_NAME = ClientComputer.Contoso.com ]  [ CLIENT_IP_ADDRESS = pri.vat.e.ip ]  [ TRANSACTION_ID = 16 ]  [ ACCESS_MASK = 0 ]  [ USERNAME = someones_name ]  [ RECORD_NUMBER = UVH@16593882 ]  [ USER_SID = S-1-5-21-xxxxxxxxx-xxxxxxxxx-xxxxxxxxx-xxxxx ]  [ ACCESS_TYPE = 5 ]  [ ACCESS_TYPE_TEXT = File/Folder Created ]  [ FORMAT_MESSAGE = User 'someones_name' Created file/folder '\\\\\\\\Isilon.Contoso.com\\\\Arbitrary\\\\File\\\\Path\\\\To\\\\2021\\\\42100030\\\\hsm\\\\0030d302_2\\xB0\\\\t'. ]  [ USER_SAM_ACCOUNT_NAME = null ]  [ USER_DISPLAY_NAME = null ]  [ USER_PRINCIPAL_NAME = null ]  [ USER_GUID = null ]  [ USER_DISTINGUISH_NAME = null ]  [ USER_OU_GUID = null ]  [ USER_DEPARTMENT = null ]  [ USER_MANAGER_NAME = null ]  [ SOURCE_NAME = null ]  [ LOG_FILE_NAME = null ]  [ KEYWORDS_NAME = null ]  [ TASK_CATEGORY_NAME = null ]  [ TASK_CATEGORY_ID = null ]  [ FILE_TYPE = Folder ]  [ SHARE_NAME = null ]  [ EXTRA_COLUMN1 = null ]  [ EXTRA_COLUMN2 = null ]  [ EXTRA_COLUMN3 = null ]  [ EXTRA_COLUMN4 = null ]  [ EXTRA_COLUMN5 = null ]  [ EXTRA_COLUMN6 = null ]  [ EXTRA_COLUMN7 = null ]  [ EXTRA_COLUMN8 = null ]  [ EXTRA_COLUMN9 = null ]  [ EXTRA_COLUMN10 = null ]  [ CONFIGURED_DOMAIN_NAME = null ]  [ NEW_PRIVILEGES_USED = null ] ", :expected_charset=>"UTF-8"}

To note, from reading other similar threads...

  • No overlapping port numbers with TCP/UDP
  • The messages are plain-text (no special encoding)
  • Logstash input is a very basic:
input {
  tcp {
    port => 5514
  }
}

I would appreciate any and all thoughts on correcting this.

Thank you in advance!

2

My guess (and it is just that) is that the line codec that the tcp input uses by default is parse the \xB0 as a hex character, which indeed would not be valid UTF-8. You could try setting the codec to use a different charset.

tcp { port => 5514 codec => line { charset => "ASCII-8BIT" } }

for example. It may still incorrectly parse it, but not produce an error.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.