Problem with splitted logstash config file in logstash 7.7

Elastic Stack Logstash
1

We're migrating from logstash 6.5.4 to 7.7.0. In 6.5.4 we had our config files all in one directory, with a naming convention like 01_. We pointed path.config to that directory and logstash concatenated the files as if it were one big file.

In 7.7 this seems to work differently. E.g. we had a file 50_filter_begin with just

filter {

And then in many files like 51_drop_some_type we had

    if [type] == "xyz" { drop { id => "xyz_drop" } }

and to close all filter files we had 89_filter_end

}

This used to work, but when I try this setup with logstash 7.7, it complains with error messages like

[2020-06-04T15:25:29,588][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"else\", \"if\", [A-Za-z0-9_-], '\"', \"'\", \"}\" at line 268, column 1 (byte 8076) after filter {\n

The whole configuration works nicely with logstash 7.7 when I cat(1) the files together before starting logstash and then point it to the concatenated file. This is my current workaround.

Did I miss something in the migration documentation or has this functionality been removed?

CU, Joe

2

This is a known issue introduced in 7.6.0. You have to wrap each individual file in

filter {
}

Depending on how you divide your filters right now that may be an almost trivial change, or a significant re-write.

3

Thanks @Badger !

The same pattern applies to outputs (we have several there, too).

It was a trivial change. Actually, we obviously prefer this format, we just didn't expect this to change