How to add field using Environment plugin in Logstash configuration file

Elastic Stack Logstash
1

Question: How to add field from Environment plugin in logstash config file. Here is my Config file
input{
stdin{
}
}

filter {

if "exception" not in [tags] {



    # example output:
    # 2016-12-16 20:43:20,535 DEBUG [CWMP-processor-6] [00D09E-0000000001:1002:C5852D7218635D7B09FE0DDE0FBE75F5:0:] c.twowire.dmc.service.PolicySvcImpl - Device matched policy 1001
    # encoder pattern (dmc/conf/logback.xml):
    # %date{ISO8601} %-5level [%thread] [%X{username}:%X{deviceId}:%X{sessionId}:%X{userInteraction}:%X{workflowName}] %logger{35} - %msg%n

    grok {
        match => {
            message => "%{DATESTAMP:timestamp} %{LOGLEVEL:level}( +)\[%{DATA:thread}\] \[%{DATA:mdc}\] %{JAVACLASS:class} - %{JAVALOGMESSAGE:logmessage}"
#message => "%{DATESTAMP:timestamp} %{LOGLEVEL:level}( +)\[%{DATA:thread}\] \[%{DATA:mdc}\] %{JAVACLASS:class} - %{GREEDYDATA:logmsg}"
        }
        # Record that this is an "log" event.
        add_tag => ["log"]
        
    }

    if "log" in [tags] {

        grok {
            match => {
                mdc => "%{DATA:username}:%{DATA:deviceId:int}:%{DATA:sessionId}:%{DATA:userInteraction:int}:%{GREEDYDATA:workflowName}"
            }
        }

        date {
            timezone => GMT
            match => [
                           # "16-12-16 21:58:20,606"
                "timestamp", "yy-MM-dd HH:mm:ss,SSS"
            ]
        }

    }


}

}

============================
So, how can I add the field to get the source name i.e; to know from which server I am getting the log file?

I tried with the following line but didn't work

add_metadata_from_env => { "SOURCE" => "SKY" } // indicates this log file is from "SKY"
But when I load the index into elastic search I could not able to see my field SKY in kibana.

Could anyone help me with this??

Thanks!

2

Which input plugins do you intend to use? Many of them (including stdin and file) already store the hostname in the host field.

If you're sure that you really need to add the hostname from an enviroment variable I suggest you just use ${SOURCE} together with add_field in your inputs.

https://www.elastic.co/guide/en/logstash/current/environment-variables.html

1 Like
3

Hi,

I wanted to use environment plugin. I will try using ${SOURCE} with add_field.

Thanks a lot !! :slight_smile:

4

Hi Magnus,

Its working fine. I did
filter{
//To add a new field
mutate{ add_field => {"source'=> "SKY"}}

// To get the host name
environment{
add_field => ["my_environment", "Hello World, from %{host}"]
}

}

Thanks for your help! :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.