Encoding in logstash configure file and there must be space before end of Exec plugin argument

Elastic Stack Logstash
1

Hi there,

I have problem using korean(utf-8) arguments in exec output.

output {
    command => "test.sh ''  '{한글과 English}'  'somthing' "   # If there is not end space '(single quote) with "(double quote), error occur!
}

Test command do echo variables.
Result is "var1 = {??? English}, var2 = something"

Korean(utf-8) is represented by ?(question mark).

There must be space before end of Exec plugin argument.
If there is not end space '(single quote) with "(double quote), error occur!

/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file is there
2

It looks like this is a problem with your script, not with Logstash?

3

When I run shell script itself, error not occur.
Below both example work.

$ /etc/logstash/test.sh '한글(utf-8)' 'blah blah'

# without space between ' and "
$ eval "/etc/logstash/test.sh '한글(utf-8)' 'blah blah'"

And is there method to represent utf-8 character in logstash configuration file?

When I send to plugin, not problem. But in exec plugin, argument is presented through logstash.

4

I can't reproduce either of the reported problems.

$ cat test.config 
input {
  exec {
    command => "echo '한글과 foo'"
    interval => 2
  }
}
output { stdout { codec => rubydebug } }
$ /opt/logstash/bin/logstash -f test.config
Logstash startup completed
{
       "message" => "한글과 foo\n",
      "@version" => "1",
    "@timestamp" => "2015-10-15T05:50:45.014Z",
          "host" => "lnxolofon",
       "command" => "echo '한글과 foo'"
}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
Logstash shutdown completed
$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
5

Plugin to Plugin is not problem.

What I do is express UTF-8 string in configure file.

like

output {
  exec {
    command => "echo '한글과 foo'"
  }
}

check out this.

6

Still not able to reproduce.

$ cat test.config 
input { stdin { } }
output {
  exec {
    command => "echo '한글과 foo'"
  }
}
$ echo some input | /opt/logstash/bin/logstash -f test.config
Logstash startup completed
한글과 foo
Logstash shutdown completed
7

Hmm...

$ cat conf.d/test.conf
# test
input { stdin { } }
output {
  exec {
    command => "echo '한글과 foo'"
  }
}

$ tail /var/log/logstash/logstash.stdout
??? foo
??? foo

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

I have no idea. maybe problem is adaptive to system.
CentOS release 6.7 (Final), logstash-1.5.4-1.noarch

8

@magnusbaeck I found this problem reproduced, when left as .stdout file!
I also work, when I start as a binary file.

I set up with rpm and run service. so /var/log/logstash/logstash.stdout is point to check.