About Comments within Logstash Plugin Config

Elastic Stack Logstash
1

Hi everyone. Recently, I am trying to add more comments to my Logstash pipeline config so that it is more understandable. However, I found that it seems comments cannot be added in-between lines of a plugin config.

For example, the following pipeline will result in the error given configuration is invalid. Reason: Expected one of [ \t\r\n], "#", "{", "}":

input {
  http {}
}

filter {
  mutate {
    copy => {
      # Copy `source` to `source2`
      source => 'source2'
      # Copy `target` to `target2`
      target => 'target2'
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

But the following pipeline is fine:

input {
  http {}
}

filter {
  mutate {
    copy => {
      # Copy `source` to `source2`
      source => 'source2'
      target => 'target2'
      # Copy `target` to `target2`
    }
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

It would be greatly appreciated if anyone could let me know what's the problem -maybe it is just me overlooking somewhere in the documentation stating the first pipeline is not a valid.

2

That is correct. This github issue includes a pointer to the formal grammar for configuration files and it clearly does not allow comments between hash entries. You can have _ before a set of attributes, or after a set of attributes, but not in a set of attributes.

One comment claims using array syntax is a workaround

mutate {
    copy => [
        "source", "source2", # Copy `source` to `source2`
        "target", "target2"  # Copy `target` to `target2`
    ]
}

and looking at the definition of an array in the grammar that does seem like it would work. Personally I do not like it as a solution.

3

Thank you for the prompt reply. I think maybe I would just divide the handlings into separate mutate plugins because of the logical disjoint (it can also avoid the case where missing in earlier fields causing latter fields not to be processed)

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.