Set a default value if nil

Elastic Stack Logstash
1

Hey,

I know it has been asked a lot, but I tried every solution and nothing works for me.
I have a field that sometimes is empty (,,) so I want to set "ebay" in that case.

Here it is how it looks like when it is nil:

    "marketplace" => nil ,

And when I set the conf file as follow it is still nil

    if [!marketplace] {
        mutate {add_field => {"marketplace" => "ebay"} }
    }

What am I doing wrong?

Thanks!!
Yaniv

2

That should be

if ! [marketplace] {
3

Thanks, it looks much better, but now the value set it this:

    "marketplace" => [
        [0] "ebay"
    ],

instead of

     "marketplace" => "ebay",
4

Doing this

if ![marketplace] {
  mutate {add_field => {"marketplace" => "ebay"} }
}

adds you the value as an array out of the pipeline? Not possible, there must be something you are omitting:

image
image844×910 86.8 KB

Please post here your whole pipeline and a sample of input (generated by the run of the pipeline without the filter section and with the stdout{} as output).

1 Like
5

Here is the full pipeline:

{
                    "@version" => "1",
                   "date/time" => "Feb 2, 2020 3:12:55 AM PST",
        "giftwrap credits tax" => 0.0,
                "order postal" => nil,
                    "quantity" => nil,
                        "type" => "Service Fee",
                    "fba fees" => 0.0,
                 "description" => "Cost of Advertising",
                        "tags" => [
        [0] "_mutate_error"
    ],
                        "path" => "/Users/yanivnuriel/logstash/data/transactions/all-14.txt",
            "shipping credits" => 0.0,
                       "total" => -122.32,
                     "message" => "\"Feb 2, 2020 3:12:55 AM PST\",12578696491,Service Fee,,,Cost of Advertising,,,,,,,,0,0,0,0,0,0,0,0,0,0,0,-122.32,0,-122.32\r",
                  "order city" => nil,
        "tax collection model" => nil,
               "product sales" => 0.0,
        "shipping credits tax" => 0.0,
      "other transaction fees" => -122.32,
           "gift wrap credits" => 0.0,
                 "order state" => nil,
     "promotional rebates tax" => 0.0,
                "selling fees" => 0.0,
         "promotional rebates" => 0.0,
           "product sales tax" => 0.0,
                 "fulfillment" => nil,
    "marketplace withheld tax" => 0.0,
               "settlement id" => "12578696491",
                       "other" => 0.0,
                    "order id" => nil,
                 "marketplace" => [
        [0] "ebay"
    ],
                        "host" => "Yanivs-MBP",
                         "sku" => nil,
                  "@timestamp" => 2020-02-02T11:12:55.000Z
}

Here is the only mutate part in config:

    if ! [marketplace] {
        mutate {add_field => {"marketplace" => "ebay"} }
    }
6

Well that's not the pipeline, that's the result of your pipeline. The pipeline is what is inside the conf file. For example, from your result I see there's a _mutate_error in the tags. Where does that come from?

Please, post here 2 things:

1 - An example of an input. It is something similar to what you posted in your previous post but without the filter section applied. So comment it out and run logstash only with the input section and the stdout{} part in the output section. This is useful for us to see how Logstash is treating your input data.

2 - Your whole pipeline (i.e. conf file). This is needed to see where you are creating that array.

1 Like
7

Thanks for the prompt respond.
I think I found the problem, in one hand the value is nil, but then I understood that if I get array than it might existed .. weird..
So I decided to change the add_field to replace:

    if ![marketplace] {
        mutate {replace => {"marketplace" => "ebay"} }
    }

Now it works..

    "selling fees" => 0.0,
    "marketplace" => "ebay",
    "type" => "Adjustment",
    "shipping credits" => 0.0,
8

Interesting

input { generator { count => 1 lines => [ '' ] } }
filter {
    ruby { code => 'event.set("someField", nil)' }
    if ! [someField] {
        mutate { add_field => { "someField" => "someValue" } }
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

produces

 "someField" => [
    [0] "someValue"
]

That is, if the field exists but is nil, then add_field adds the single value as an array. I would argue that that is a bug.

2 Likes
9

Thanks for you time and guidelines

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.