# Aggregation error - nil can't be coerced into Fixnum

**URL:** <https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678>\
**Category:** Logstash\
**Created:** [February 16, 2019, 10:09am UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678 "2019-02-16T10:09:43Z")\
**Posts on this page:** 18\
**Page:** 2

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 12:43am UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/21 "2019-02-21T00:43:28Z")

</div>

I think you need to take elasticsearch and kibana out of the equation and just run with

```
output { stdout { codec => rubydebug } }

```

See if the events that logstash generates are what you expect. Then adjust either the events or your expectations. Start off with a really simple configuration like the one I posted [earlier](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/11?u=badger) in the thread (swapping out the json filter for your csv filter).

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 2:10pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/22 "2019-02-21T14:10:14Z")

</div>

I did it and it is not working. **I am not getting the final event**.  
Just Need to mention that I am running version 6.6.0 on windows machine.

**My Input file:** (foo.json)

```
{ "PROBLEM" : 100 }
{ "PROBLEM" : 101 }
{ "PROBLEM" : 102 }
{ "PROBLEM" : 103 }
{ "PROBLEM" : 104 }
{ "PROBLEM" : 105 }
{ "PROBLEM" : 106 }

```

**Input:**

```
input {
       file {
             path => "C:/kpi/foo.json"
             start_position => "beginning"
             sincedb_path => "NUL"
       }
}

```

**My filter:**

```
filter {
     if [path] == "C:/kpi/foo.json"
     {
         json { source => "message" }
         aggregate {
                task_id => "%{PROBLEM}"
                code => "
                     map['FooJustTesting'] ||= 0; map['FooJustTesting'] += 1
                "
                push_map_as_event_on_timeout => true
                inactivity_timeout => 2
         }
     }
}

```

**My output:**

```
output {
     if [path] == "C:/kpi/foo.json"
     {
         stdout{}
     }
}

```

**The stdout results:**

```
   {
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 100 }\r",
    "@timestamp" => 2019-02-21T14:02:00.457Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 100
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 104 }\r",
    "@timestamp" => 2019-02-21T14:02:00.522Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 104
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 102 }\r",
    "@timestamp" => 2019-02-21T14:02:00.521Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 102
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 103 }\r",
    "@timestamp" => 2019-02-21T14:02:00.522Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 103
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 101 }\r",
    "@timestamp" => 2019-02-21T14:02:00.518Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 101
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 105 }\r",
    "@timestamp" => 2019-02-21T14:02:00.523Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 105
}
{
          "host" => "SHARONSA03",
       "message" => "{ \"PROBLEM\" : 106 }\r",
    "@timestamp" => 2019-02-21T14:02:00.523Z,
          "path" => "C:/kpi/foo.json",
      "@version" => "1",
       "PROBLEM" => 106
}

```

Thanks  
Sharon.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 2:12pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/23 "2019-02-21T14:12:48Z")

</div>

You have "--pipeline.workers 1", right?

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 2:18pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/24 "2019-02-21T14:18:36Z")

</div>

Don't know. How to check?

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 2:19pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/25 "2019-02-21T14:19:06Z")

</div>

I have default

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 2:21pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/26 "2019-02-21T14:21:06Z")

</div>

```
# This defaults to the number of the host's CPU cores.
#
# pipeline.workers: 2

```

It is commented in the logstash.yml

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 2:28pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/27 "2019-02-21T14:28:09Z")

</div>

Uncomment that and set it to 1.

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 2:58pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/28 "2019-02-21T14:58:00Z")

</div>

```
{
    "@timestamp" => 2019-02-21T14:49:36.496Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 100,
       "message" => "{ \"PROBLEM\" : 100 }\r",
      "@version" => "1",
          "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.558Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 101,
       "message" => "{ \"PROBLEM\" : 101 }\r",
      "@version" => "1",
            "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.558Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 102,
       "message" => "{ \"PROBLEM\" : 102 }\r",
      "@version" => "1",
            "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.574Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 103,
       "message" => "{ \"PROBLEM\" : 103 }\r",
      "@version" => "1",
           "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.574Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 104,
       "message" => "{ \"PROBLEM\" : 104 }\r",
      "@version" => "1",
           "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.574Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 105,
       "message" => "{ \"PROBLEM\" : 105 }\r",
      "@version" => "1",
               "path" => "C:/kpi/foo.json"
}
{
    "@timestamp" => 2019-02-21T14:49:36.574Z,
          "host" => "SHARONSA03",
       "PROBLEM" => 106,
       "message" => "{ \"PROBLEM\" : 106 }\r",
      "@version" => "1",
            "path" => "C:/kpi/foo.json"
}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 2:59pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/29 "2019-02-21T14:59:24Z")

</div>

> [@ssasporta](#):
>
> output { if [path] == "C:/kpi/foo.json" { stdout{} } }

The aggregate event will not have a path field because you did not add it to the map. Remove this conditional.

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 3:11pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/30 "2019-02-21T15:11:30Z")

</div>

Good... We are getting close to a solution.....

**Now I see that in the end of the stdout:**

```
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.247Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}
{
    "FooJustTesting" => 1,
        "@timestamp" => 2019-02-21T15:04:44.262Z,
          "@version" => "1"
}

```

Shouldn't I get `FooJustTesting" => 6` ?

`map['FooJustTesting'] ||= 0; map['FooJustTesting'] += 1`

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 3:13pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/31 "2019-02-21T15:13:34Z")

</div>

Who to add a path to the map?

The path is the way I differ between the various inputs.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 3:21pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/32 "2019-02-21T15:21:21Z")

</div>

> [@ssasporta](#):
>
> Shouldn't I get `FooJustTesting" => 6` ?

No. There are 7 different map entries, one for each value of PROBLEM. Add

```
timeout_task_id_field => "PROBLEM"

```

and you will see that. If you want the path to be on the event then include

```
map['path'] = event.get('path')

```

to the code in the aggregate filter.

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 3:33pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/33 "2019-02-21T15:33:43Z")

</div>

> [@Badger](#):
>
> map['path'] = event.get('path')

**The "path" is working !**

The **FooJustTesting** is still 1. My purpose it to aggregate the values in the PROBLEM field.  
Let's assume that the field contains duration. I am trying to aggregate the duration, to have total duration of all the events.

```
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.405Z,
           "PROBLEM" => "100",
              "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.441Z,
           "PROBLEM" => "101",
         "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.441Z,
           "PROBLEM" => "102",
              "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.441Z,
           "PROBLEM" => "103",
            "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.442Z,
           "PROBLEM" => "104",
             "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.442Z,
           "PROBLEM" => "105",
            "path" => "C:/kpi/foo.json"
}
{
    "FooJustTesting" => 1,
          "@version" => "1",
        "@timestamp" => 2019-02-21T15:27:22.442Z,
           "PROBLEM" => "106",
           "path" => "C:/kpi/foo.json"
}

```

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 3:36pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/34 "2019-02-21T15:36:14Z")

</div>

I think I understand  
Let me try

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 4:23pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/35 "2019-02-21T16:23:23Z")

</div>

**This is now my aggregation:**

```
filter {
     if [path] == "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json"
     {
         json { source => "message" }
         aggregate {
                task_id => "%{PROBLEM}"
                code => "
                     map['FooJustTesting'] ||= 0; map['FooJustTesting'] += event.get('PROBLEM')
                     map['path'] = event.get('path')
                     event.set('TotalDuration', map['FooJustTesting'])
                "
                timeout_task_id_field => "PROBLEM"
                push_map_as_event_on_timeout => true
                inactivity_timeout => 2
         }
     }
}

```

I expected to **TotalDuration** to include the total values: (100+101+....etc)

**The current output results is:**

```
{
          "PROBLEM" => 100,
    "TotalDuration" => 100,
         "@version" => "1",
       "@timestamp" => 2019-02-21T15:43:56.254Z,
             "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "message" => "{ \"PROBLEM\" : 100 }\r",
             "host" => "SHARONSA03"
}
{
          "PROBLEM" => 101,
    "TotalDuration" => 101,
         "@version" => "1",
       "@timestamp" => 2019-02-21T15:43:56.337Z,
             "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "message" => "{ \"PROBLEM\" : 101 }\r",
             "host" => "SHARONSA03"
}
{
          "PROBLEM" => 102,
    "TotalDuration" => 102,
         "@version" => "1",
       "@timestamp" => 2019-02-21T15:43:56.337Z,
             "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "message" => "{ \"PROBLEM\" : 102 }\r",
             "host" => "SHARONSA03"
}
etc.....
{
    "FooJustTesting" => 100,
        "@timestamp" => 2019-02-21T15:44:03.831Z,
           "PROBLEM" => "100",
              "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "@version" => "1"
}
{
    "FooJustTesting" => 101,
        "@timestamp" => 2019-02-21T15:44:03.862Z,
           "PROBLEM" => "101",
              "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "@version" => "1"
}
{
    "FooJustTesting" => 102,
        "@timestamp" => 2019-02-21T15:44:03.862Z,
           "PROBLEM" => "102",
              "path" => "C:/Users/sharonsa/Work/GSS/VIVO/kpi/foo.json",
          "@version" => "1"
}

```

etc....

Where is my current mistake?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 21, 2019, 4:49pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/36 "2019-02-21T16:49:23Z")

</div>

> [@ssasporta](#):
>
> task\_id =\> "%{PROBLEM}"

An aggregate filter will aggregate data for a particular task\_id. In your example, every record has a different task\_id value.

Going back to your original data set. If you want to aggregate data from different records for the same incident ticket, then use the incident id as the task id. If you want to aggregate across all records, use a constant as the task\_id. For example,

```
    mutate { add_field => { "[@metadata][task_id]" => "1" } }
    aggregate {
        task_id => "%{[@metadata][task_id]}"
        code => " map['JustTesting'] ||= 0; map['JustTesting'] += 1 "
        push_map_as_event_on_timeout => true
        timeout_code => 'event.set("[@metadata][wanted]", true)'
        inactivity_timeout => 2
    }
    if ! [@metadata][wanted] { drop {} }

```

will get you

```
{
"JustTesting" => 7,
   "@version" => "1",
 "@timestamp" => 2019-02-21T16:48:04.015Z
}

```

---

<div class="post-metadata">

**Author:** ![ssasporta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ssasporta/32/13695_2.png) [@ssasporta](https://discuss.elastic.co/u/ssasporta)\
**Post date:** [February 21, 2019, 5:13pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/37 "2019-02-21T17:13:34Z")

</div>

> [@Badger](#):
>
> mutate { add\_field =\> { "[@metadata][task\_id]" =\> "1" } } aggregate { task\_id =\> "%{[@metadata][task\_id]}" code =\> " map['JustTesting'] ||= 0; map['JustTesting'] += 1 " push\_map\_as\_event\_on\_timeout =\> true timeout\_code =\> 'event.set("[@metadata][wanted]", true)' inactivity\_timeout =\> 2 } if ! [@metadata][wanted] { drop {} }

Working!!!!

```
{
        "@timestamp" => 2019-02-21T17:11:54.205Z,
    "FooJustTesting" => 721,
              "path" => "C:/kpi/foo.json",
          "@version" => "1"
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 21, 2019, 5:13pm UTC](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678/38 "2019-03-21T17:13:35Z")

</div>

Closed on March 21, 2019, 5:13pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/aggregation-error-nil-cant-be-coerced-into-fixnum/168678.md?page=1)