Support hebrew characters in logs

[orlevy08 (Orr Levy)]

Hi All,
I have an issue with logstash parsing using grok:
When some hebrew characters are placed in the log the result i see is ?????? instead of the user name (for example: if the user name is בלהבלה i get ??????)

Is there a way to sort it out?
I've tried to use the following filter with no avail:
codec => plain{
charset => "UTF-8"
}

to better explain, this is how my input section looks like:
file{
path=> "C:\Patches\532915\Test999.txt"
start_position => "beginning"
codec => plain{
charset => "UTF-16"
}
}

and output:
output {
stdout { codec => rubydebug}
}

Thank you !

[magnusbaeck (Magnus Bäck)]

What character set does the log have?

[orlevy08 (Orr Levy)]

the Input log is UTF-8 however, we are going to use splunk in the future.
Is there a way to force the input to be UTF-8?

[magnusbaeck (Magnus Bäck)]

Is there a way to force the input to be UTF-8?

I'd expect charset => "UTF-8" to work. If it doesn't I don't know what's up.

[orlevy08 (Orr Levy)]

In a strange way, we have managed to solve this issue by adding:
codec => plain{
charset => "Windows-1252"
}

instead of UTF-8 (although the input is in UTF-8)....
Thank you !