Can use variables for output influxdb db and measurement field?

Elastic Stack Logstash
1

Hi
I'm wondering if a configuration like this can work or not.

filter {
  mutate {
    if [topic] == "xxxx" {
      add_field => { "db" => "test2", "measurement" => "access_logs" }
    } else {
      add_field => { "db" => "unknown_topics", "measurement" => "unknown_logs" }
    }
  }
}

output {
  influxdb {
    id => "mydb"
    host => "localhost"
    db => "%{[db]}"
    measurement => "%{[measurement]}"
  }
}
2

The influxdb output calls sprintf for both db and measurement options, so I would expect that to work.

3

@Badger
Thank you and sorry for the late reply.
Actually, it didn't work as expected. They were just used as normal strings and Logstash was trying to connect to a database named %{[db]} in Influxdb.

4

@Badger
I found the reason. It's a known bug and the fix has not been merged yet. (2 years ago PR... I don't know what to say... :roll_eyes:)

5

Oh yes, I had forgotten about that. (I definitely knew about it once because there is a comment from me on one of the linked issues :slight_smile: )

1 Like
6

@Badger
Yup, thank you~

7

Hi @Badger
Sorry for pinging you again.
If we want to use variables as retention_policy, do you know how to approach this?
I'm thinking the same way as database, but it seems only database is passed to flush function. So, I'm not sure if we can extend the flush function with more parameters.

8

If I had to do it I would probably change this line to use event.sprintf(@retention_policy) instead of just @retention_policy.

1 Like
9

@Badger
Thank you. I've already tried that but it didn't work.
I think it's because the passed retention_policy variable has to be passed to buffer_receive first.

I also tried to add this to this line.

@retention_policy = event.sprintf(@retention_policy)

But this can work for the first retention policy being passed only. The different values with the following events will use the last value.

10

Sorry, I am not a logstash developer and I have never used influxdb. It sounds like your guesses are ahead of mine at this point.

11

No worries. Thank you for your great help.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.