Quotes in config files v2.3.0

Elastic Stack Logstash
#1

Hi all,
When trying out my config on a windows laptop this seemed to work fine (v2.2.0 by the way):
input { s3 { credentials => ["XXXXXXXXXXXXXXXXXXXXXXXXXXXXX","XXXXXXXXXXXXXXXXXXXXXX"] bucket => "XXXXXX-logqueue" region => "eu-west-1" prefix => "logs/" codec => "json"{ charset => "ISO-8859-1"} type => "s3_logqueue" backup_to_bucket => "XXXXXX-archivedlogs" delete => "true" } }

But when i ran it on ubuntu, using v2.3.0 i had to remove the quotes around "json" otherwise it said plugin "json" not found. Is this likely to be a windows issue or could it be that the new version is taking the values literally and not ignoring the quotes?

#2

Just tested 2.3.0 on windows and got the same error:
"Couldn't find any codec plugin named '\"json\"'. Are you sure this is correct? Trying to load the \"json\" codec plugin resulted in this error: no such file to load -- logstash/codecs/\"json\"", :level=>:error}

#3

Try 2.3.1, there are a few issues with 2.3.0.

#4

Hi Mark,
Probably a dumb question but where do i get 2.3.1 from?

#5

Github releases page :slight_smile:

#6

Thanks :slight_smile: I'll give it a go

#7

2.3.1 is due to be released today :slight_smile:

#8

Note, that there may be some delays, but today is our target.

#9

I'll try again once its released, as I might have built it wrong (always used the packaged version before), but i ran "rake bootstrap", then from the bin folder:
./logstash -f /etc/logstash/conf.d --configtest

and got:
The given configuration is invalid. Reason: Couldn't find any codec plugin named '"json"'. Are you sure this is correct? Trying to load the "json" codec plugin resulted in this error: no such file to load -- logstash/codecs/"json" {:level=>:fatal}

So I think its still reading the quotes literally, not a major issue but it seems a bit odd :slight_smile:
Cheers!

#10

Yes, this is a bug. Looking into it I can confirm this will have to wait for 2.3.2 or later, it will not be fixed in 2.3.1.

I've opened up an issue here https://github.com/elastic/logstash/issues/5061

#11

Thanks Andrew, I can just remove the quotes for now, it's not too much of a workaround

#12

the charset could be an issue if it isn't reading that. Maybe I'll drop back to a 2.2 version for now :smile:

#13

@Alex_6 @StephenGoodall please note that the https://github.com/elastic/logstash/releases are just repo tar.gz and not a proper release package. It is basically the same a cloning the repo at the release tag and you will have to build logstash to have something usable.

#14

FYI 2.3.1 is now released, see https://www.elastic.co/blog/logstash-2.3.1-and-2.2.4-released - in particular anyone who installed 2.3.0 should absolutely upgrade to 2.3.1.

Note that there is no change for the quoting, so @StephenGoodall removing the quotes around the codec name is the right thing to do at this point!

Colin

#15

thanks @colinsurprenant
I've upgraded to 2.3.1 for some testing, i will remove the quotes around the codec, but do you know if the charset specification will be picked up?
so it will be like this:

input { s3 { credentials => ["XXXXXXXXXXXXXXXXXXXXX","XXXXXXXXXXXXXXXXXXXXXXX"] bucket => "XXX-logqueue" region => "eu-west-1" prefix => "logs/" codec => json { charset => "ISO-8859-1"} type => "s3_logqueue" backup_to_bucket => "XXX-archivedlogs" delete => "true" } }

#16

@StephenGoodall yes, charset => "ISO-8859-1" is totally a valid syntax and if it doesn't work this is a bug.

#17

Great, Thanks Colin

#18