Problem when log stash is executed as a service (Ubuntu)

[caleneo (Pablo Serrano)]

Hello, I'm having several issues trying to execute it as a service. Using the unzipped files everything is OK but when I copy the configuration file into the conf.d/ directory I get always the same error:

[2017-01-07T02:26:22,267][INFO ][logstash.runner ] Created final config by merging config string and config path {:path=>"/etc/logstash/conf.d"}
[2017-01-07T02:26:22,279][FATAL][logstash.runner ] The given configuration is invalid. Reason: Expected one of #, input, filter, output at line 1, column 1 (byte 1) after

My config file is a very simple one, like this:

input { stdin { } }
output {
stdout {
codec => rubydebug
}
file {
path => "/tmp/sample.log"
}
}

I've already checked it using hex dump -C and I got the following result:

00000000 69 6e 70 75 74 20 7b 20 73 74 64 69 6e 20 7b 20 |input { stdin { |
00000010 7d 20 7d 0a 6f 75 74 70 75 74 20 7b 0a 20 20 73 |} }.output {. s|
00000020 74 64 6f 75 74 20 7b 20 0a 20 20 20 20 63 6f 64 |tdout { . cod|
00000030 65 63 20 3d 3e 20 72 75 62 79 64 65 62 75 67 0a |ec => rubydebug.|
00000040 20 20 7d 0a 20 20 66 69 6c 65 20 7b 0a 20 20 20 | }. file {. |
00000050 20 70 61 74 68 20 3d 3e 20 22 2f 74 6d 70 2f 73 | path => "/tmp/s|
00000060 61 6d 70 6c 65 2e 6c 6f 67 22 0a 20 20 7d 0a 7d |ample.log". }.}|
00000070 0a |.|
00000071

which means that the file is OK... any ideas? permissions are also OK because the file is read. I'm quite lost at this moment...

Thanks

[eperry (Ed)]

do you have any other files in the conf.d directory? As logstash default config will load every file regardless of its extension

I was going to say dos 2 unix your file but the hex dump seems to be right. as it says it has an issue with line 1 char 1

what java version are you running?

[caleneo (Pablo Serrano)]

Thanks for the reply!

There are no other files in the conf.d folder. The java version is Oracle Java 1.8 downloaded using apt-get.

I'm quite confused because the stand-alone jar explicitly indicates that the configuration file is OK and runs without problems. I suspect that it could be related with file permissions...

Any other ideas?

Regards,
Pablo

[eperry (Ed)]

well, by default on rhel the logstash agent runs as logstash user /etc/sysconfig/logstash

There is a userid of LS_USER I believe, or you could try changing the permissions of the
chmod +r /etc/logstash/conf.d/*.conf

That would deal with any permission problems reading the log file.

[caleneo (Pablo Serrano)]

Sorry for the big delay...

The problem persists, even in different systems... I'm doing some tests using my Mac and I'm also having issues but this time between Filebeats and Logstash....

Is it possible that the problems are related with java-8? I'm not using the openjdk...

[eperry (Ed)]

nope there is no problem with jdk1.8, its is probably something you are doing or assuming is happening. There is probably a hidden file or your looking in the wrong location for the config file

you could try running a strace on logstash to see what files it is opening

strace -e open -ff /opt/logstash/bin/logstash -f /etc/logstash/conf.d

and see if you can find what file it is loading.

[theuntergeek (Aaron Mildenstein)]

I'm more inclined to believe it's an encoding issue. Are your config files UTF-8? Or something else? What are the newline characters?

[Rinat]

Has anything been changed in the directory structure for logstash from the time when /etc/sysconfig had logstash file?
I'm unable to locate /etc/sysconfig/logstash. I do see elasticsearch in the directory but not logstash.

Thanks