Influxdb ( Victoria Metrics ) Error + Filtering Logstash ( remove - split )

Here is only one example from my JSON file >>

{"host":"ABCDEFASD","groups":["ABVD","TEST"],"applications": ["NETWORK"],"itemid":143172,"name":"Operational status of interface Se0/1/0:17","clock":1604283792,"ns":926563322,"value":2}

AND I want to change this json file to metrics format:

metrics{host="ABCDEFASD", groups="ABVD",groups="TEST",applications="NETWORK",name="Operational status of interface Se0/1/0:17", "itemid":143172}

Can you help me with this? What should I follow with my config file?
1-Problem : Remove > value, ns, clock
2-Problem : Separate > "groups":[ABVD","TEST"] groups="ABVD",groups="TEST"
3-Problem : metrics { }
4-Problem : Influxdb, saving it on Victoria Metrics

I am connecting to SSH server with putty >> CentOS8 Linux Machine

`
filter {

    mutate {

            remove_field => ["value", "ns", "clock"]

    }

    mutate {

            split => { "groups" => "," }

    }
      ruby {

            code =>"

            event.set("[metrics][host]",event.get("host"));

            event.remove("host")"

    }

   }

output {

    stdout {

            codec => rubydebug

            }

    influxdb  {

            host => "10.92.161.1"

            port => 4288

            db => "metrics"

            codec => "json"

            }

    }

`

influxdb output plugin is not bundled by default with logstash,
it is easy to install by running bin/logstash-plugin install logstash-output-influxdb

This command doesn't work for CentOS 8.

  1. May be you should run it from /usr/share/logstash
    Check it here the structure of directory of logstash

  2. If you use codec => "json" when you need to tranform your json to metrics format
    You can simply remove unecessary fields and then send the json directly

What should I change with my configuration file for the metrics part? [ {host=...} to metrics{host=..} ]
And remove - separating part is correct, do you think?
Because of influxdb problem, I am not able to check the output.

Continue with your example, install the influxdb output plugin and remove this

mutate { split => { "groups" => "," }}

Try and share your feedback

I just installed influxdb and solved the one of the output problem.
Now I am getting this error :
Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console

Why did you offer to remove that filtering part?
I want this with that >>
"groups":["ABVD","TEST"]"
groups="ABVD",groups="TEST"

Sorry, I'm not aware of the format that should be used by influxdb
I see that the influxdb ouput plugin have a parameter called codec that can be setted to plain or json and you are using codec => "json", that let me think that influxDB will accept directly the json payload sent by logstash ... so in my opinion you can try to send json without splitting it :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.