Convert normal logstash output to json output for adx ingestion

Elastic Stack Logstash
1

I have been working on transferring Palo Alto firewall logs(syslog format) to ADX. To achieve this, I developed grok filters and incorporated kv and mutate filters as well. However, I encountered an issue where the output was not in the desired JSON format. To address this, I applied the 'codec => json' configuration within the Kusto plugin, and it successfully resolved the problem. Nevertheless, I'm uncertain if this approach is considered a best practice or if it might potentially cause any complications."

output {
    kusto {
            codec => json
            path => "/tmp/kusto/%{+YYYY-MM-dd-HH-mm-ss}.txt"
            ingest_url => "https://ingest-<cluster name>.kusto.windows.net/"
            app_id => "<application id>"
            app_key => "<application key/secret>"
            app_tenant => "<tenant id>"
            database => "<database name>"
            table => "<target table>" # logs as defined above
            json_mapping => "<mapping name>" # basicmsg as defined above
    }
}
2

If you want JSON formatted output then it is appropriate to use a json codec.

3

Indeed, my ADX plugin specifically requires input in JSON format, which is why I require JSON formatting. Initially, I considered using a filter such as {json { source => syslog_message }} to achieve this. However, I found that implementing such a filter was making things more complex and convoluted. Therefore, I opted for the simpler solution of utilizing the codec => json configuration.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.