How to create a custom document

Elastic Stack Logstash
1

Hello
I was able to configure the following:
From the json document I have this:

"MyValue": "Key value based on a specific field",

"host": {
"name": "hostname"
},

"tags": [
"beats_input_codec_plain_applied",
],

... SNIP ...

I would like to create some similar instead of having key:value like this

"MyValue" : {
"name": "content from the specific field"
}

Or something like this:

"MyTags": [
"product_tag",
"another_custom_tag",,
],

Did I myself clear? is that possible to do with logstash?
Thanks
Regards

2

You can use the add_field and add_tag common options inside any filters (and most if not all of the input and output plugins if I recall correctly). With these you can construct any shape of hash you wish at the "output" side of logstash.

If you want a little more detail, it would be helpful if you share some of your input data and your logstash configuration. We could then possibly offer other ways to achieve your end goal.

3

@ben.west Hello, thanks for your help
I have a log which contains something like this:

20191030102428 ,XX,CONDITION_COLOR,SYSTEM~JAVA~hostname_SYSTEMID_00,Errors message

First column is a time stamp, XX can be an alert condition , condition_color can be RED, GREEN or something, the rest of the columns are descriptions about the system triggering the alert, so far
so good with that.

Then I have this filter to rename default name column names.

filter {
	csv { autodetect_column_names => false }
		mutate {
		 rename => { "column4" => "Description" }
		 rename => { "column2" => "Severity" }
		 rename => { "column3" => "Status" }
		 rename => { "column5" => "Alert_Description" }
		 rename => { "column1" => "Alert_TimeStamp" }
		 
	  }
}

With that configuration I get for example, on the document root:
Status: GREEN

But I would like to have something like this:

Status": [
"GREEN"
],
Or

"Status_Tag": {
"name": "GREEN""
},

Thanks
Regards

4

Hi @Kernel_Panic,

You can use the nested notation to create/rename columns as you wish like this:
rename => { "column3" => "[status_tag][name]" }

Which will create a hash in the output object:

{
             "@version" => "1",
                 "path" => "/tmp/test.csv",
              "message" => "20191030102428 ,XX,CONDITION_COLOR,SYSTEM~JAVA~hostname_SYSTEMID_00,Errors message",
             "Severity" => "XX",
          "Description" => "SYSTEM~JAVA~hostname_SYSTEMID_00",
    "Alert_Description" => "Errors message",
      "Alert_TimeStamp" => "20191030102428 ",
                 "host" => "d506bfbd5ab6",
           "@timestamp" => 2019-11-11T11:01:02.357Z,
           "status_tag" => {
        "name" => "CONDITION_COLOR"
    }
}

The config reference for events is a great place to get started using and understanding nested fields.

Thanks,
Ben

5

@ben.west thank you very much.!!
Regards

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.