Problem with splitted logstash config file in logstash 7.7

We're migrating from logstash 6.5.4 to 7.7.0. In 6.5.4 we had our config files all in one directory, with a naming convention like 01_. We pointed path.config to that directory and logstash concatenated the files as if it were one big file.

In 7.7 this seems to work differently. E.g. we had a file 50_filter_begin with just

filter {

And then in many files like 51_drop_some_type we had

    if [type] == "xyz" { drop { id => "xyz_drop" } }

and to close all filter files we had 89_filter_end

}

This used to work, but when I try this setup with logstash 7.7, it complains with error messages like

[2020-06-04T15:25:29,588][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"else\", \"if\", [A-Za-z0-9_-], '\"', \"'\", \"}\" at line 268, column 1 (byte 8076) after filter {\n

The whole configuration works nicely with logstash 7.7 when I cat(1) the files together before starting logstash and then point it to the concatenated file. This is my current workaround.

Did I miss something in the migration documentation or has this functionality been removed?

CU, Joe

This is a known issue introduced in 7.6.0. You have to wrap each individual file in

filter {
}

Depending on how you divide your filters right now that may be an almost trivial change, or a significant re-write.

Thanks @Badger !

The same pattern applies to outputs (we have several there, too).

It was a trivial change. Actually, we obviously prefer this format, we just didn't expect this to change

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.