Logstash-input-file scans non latin character file name cause InvalidPathException and stopped working

[lixplor]

Hi,

I'm using Logstash 6.8.23 with file input plugin 4.1.18 on a JRE 8 env. OS is suse11 and system default locale is zh_CN.gbk.

When Logstash collecting logs, it reached a file with non-latin name. eg. 你好.xml. Then Logstash shows below errors in logstash-plain.log and stopped collecting:

[2022-04-01T01:00:00,00][ERROR][logstash.pipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::File start_position=>"beginning", path=>... check_archive_validity=>false>
  Error: Malformed input or input contains unmappable characters: /home/user/???????.xml
  Exception: Java::JavaNioFile::InvalidPathException
  Stack: sun.nio.fs.UnixPath.encode(UnixPath.java:147)
  ...
[2022-04-01T01:00:00,00][INFO][filewatch.observingtail ] QUIT - closing all files and shutting down.
[2022-04-01T01:00:00,00][INFO][filewatch.observingtail ] QUIT - closing all files and shutting down.

I checked Logstash that it was running with -Dfile.encoding=UTF-8. But it seems cannot work with UTF-8 character file path.

This problem happens on Logstash 6.8.23. When I was using Logstash 6.0.0. There was no such error. So I think there're some encoding config I should make?

[lixplor]

Hi guys,

I figured it out. The reason is surely system encoding does not support filename characters.
To resolve it, you can ether change system encoding, or just add a jvm parameter -Dsun.jnu.encoding=UTF-8