How to add nested log data or replace string values with nested log data on logstash.conf?

Elastic Stack Logstash
1

I want to do the data modification with logstash.conf like (see "host" key):

From:
{
"host": "192.168.1.1",
"CompanyNo": "9999999999998",
"@timestamp": "2019-07-12T05:19:40.046Z",
}

To:
{
"host": {"name": "192.168.1.1"},
"CompanyNo": "9999999999998",
"@timestamp": "2019-07-12T05:19:40.046Z",
}
or
{
"host": {"name": "any_strings"},
"CompanyNo": "9999999999998",
"@timestamp": "2019-07-12T05:19:40.046Z",
}

I tried the following commands at "filter" part on logstash.conf, but none of them succeeded:
filter {
mutate {
replace => { "host" => {"name" => "dummy_not_use_this_field"} }
}
}

filter {
mutate {
add_field => { "host" => {"name" => "dummy_not_use_this_field"} }
}
}

I got only the following stdoutput on the terminal with these two filter patterns:
[2019-07-12T20:00:34,613][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, :thread=>"#<Thread:0x7d2be381 run>"}
[2019-07-12T20:00:34,624][INFO ][logstash.inputs.file ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sincedb_c700f097b3a0f1f67921ca2ef1769d82", :path=>["/srv/logstash/srv/logstash*/**/ls_rcv-*.txt"]}
[2019-07-12T20:00:34,625][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-07-12T20:00:34,626][INFO ][filewatch.observingread ] START, creating Discoverer, Watch with file and sincedb collections
[2019-07-12T20:00:34,644][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}

I made sure the following code successfully displayed the input log data on screen (with "host" => "dummy_not_use_this_field") when I defined "output" as stdout{ coded => rubydebug}. So I think the other part of my program would be correct.
filter {
mutate {
replace => { "host" => "dummy_not_use_this_field" }
}
}

Could you tell me how to achieve the aforementioned data modification?

2

You could do it using

    mutate { rename => { "[host]" => "[@metadata][host]" } }
    mutate { add_field => { "[host][name]" => "%{[@metadata][host]}" } }
1 Like
3

Thank you for your reply! Your suggestion formatted the data as I expected like:

      "host" => {
    "name" => "bf12dd50983d"
},

I still have error to input the data to my elastic search, the question I posted here itself was solved. I appreciate it.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.