# How to write while Loops inside plugins or filters?

**URL:** <https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426>\
**Category:** Logstash\
**Created:** [June 5, 2019, 5:13pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426 "2019-06-05T17:13:59Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 5, 2019, 5:13pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/1 "2019-06-05T17:13:59Z")

</div>

Hi,

Is there any way to write loops inside filters or outputs plugins?

For a certain message

`{"time"=100,"name"="test"}`

I want to implement something like below:

```
c=0,
while c<=10
time=time+c
c=c+1

if c=10
reset to c=0

```

For every message the time needs to get updated until the count reaches 10 and the count needs to reset to 0.  
where can i write this logic ? Filters ? outputs ?  
I need every message i send to the input filter to get this udpdated before it transforms as output.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2019, 6:07pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/2 "2019-06-05T18:07:55Z")

</div>

You could implement that in a ruby filter or an aggregate filter.

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 5, 2019, 9:03pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/3 "2019-06-05T21:03:36Z")

</div>

I am new to ruby and I did write code to do it.

```
ruby {
    code => "event.set('timestamp', event.get('timestamp_object').to_i * 1000);
    event.set('time',event.get('timestamp_object').to_i*100000000);
    count=0
    while count<=10 do
    puts count
    time=(time+count)
    if count==10
    count=0
    puts 'resetting count to 0'
    end
    end"
  }

```

I did see the below error though

```
[2019-06-05T16:58:38,521][ERROR][logstash.filters.ruby] Ruby exception occurred: undefined method `+' for nil:NilClass
[2019-06-05T16:58:38,523][ERROR][logstash.filters.ruby] Ruby exception occurred: undefined method `+' for nil:NilClass

```

and this as the message:

```
{
          "tags" => [
        [0] "_grokparsefailure",
        [1] "_rubyexception"
    ],
    "@timestamp" => 2019-06-05T20:58:38.226Z,
       "message" => "/-/depd-1.1.2.tgz|HTTP/1.1|200|9026",
          "host" => "SI-M-CG5",
      "@version" => "1",
          "time" => 0,
          "path" => "/testing/test-arti.log",
     "timestamp" => 0
} 

```

Do you know what might be causing this or if my code is faulty ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2019, 9:11pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/4 "2019-06-05T21:11:52Z")

</div>

> [@naveenrt23](#):
>
> time=(time+count)

You never initialized the time variable, so it is nil, and you cannot add to nil.

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 5, 2019, 9:31pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/5 "2019-06-05T21:31:41Z")

</div>

> [@naveenrt23](#):
>
> event.set('time',event.get('timestamp\_object').to\_i\*100000000);

I was hoping this would be sufficient to intialize it

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 5, 2019, 9:57pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/6 "2019-06-05T21:57:17Z")

</div>

No, that adds a field to the event, it does not create a ruby variable. You could add

```
time = event.get('time')

```

after that statement.

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 5, 2019, 10:14pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/7 "2019-06-05T22:14:22Z")

</div>

Thanks ! that did fix the error but i think my code is written wrong for the requirements. It's actually going to an infinite loop when i want to execute and update only once for every execution and i am getting a grok parse failure

```
code => "event.set('timestamp', event.get('timestamp_object').to_i * 1000);
    event.set('time',event.get('timestamp_object').to_i*100000000);
    count=0
    if count<=10
    count=count+1
    puts count
    time=event.get('time')
    puts 'time'
    time=(time+count)
    puts time
    end
    if count==10
    count=0
    puts 'resetting count to 0'
    end"

```

and this is the ouptut

> 1  
> time  
> 1  
> {  
> "message" =\> "-npm-npm/gzip-size/-/gzip-size-3.0.0.tgz|HTTP/1.1|200|2015",  
> "tags" =\> [  
> [0] "\_grokparsefailure"  
> ],  
> "timestamp" =\> 0,  
> "@version" =\> "1",  
> "host" =\> "SI-M-C5",  
> "@timestamp" =\> 2019-06-05T22:44:00.693Z,  
> "time" =\> 0,  
> "path" =\> "/testing/test-arti.log"  
> }

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 6, 2019, 12:26am UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/8 "2019-06-06T00:26:36Z")

</div>

I do not really understand the requirements of your use case but this might demonstrate why this is really not a good fit for logstash.

This code takes the time field from an event and adds a number from 0 to 10 to it.

```
input { generator { count => 100 lines => ['{ "time": 100}'] } }
filter {
    json { source => "message" }
    ruby {
        init => '
            @c = 0
        '
        code => '
            event.set("timex", @c + event.get("time"))
            @c += 1
            if @c >= 10
                @c = 0
            end
        '
    }
}

output { stdout { codec => plain { format => "%{timex}
" } } }

```

If you run that with "-w 3 -b 3" then it results in a stream of numbers that starts with

```
100
101
102
106
107
108
109
100
101
102
103
104
105
106
107
103
104

```

The reason I set "-b 3" is so that I do not have to post a thousand lines of data to demonstrate the issue of different threads interacting. If you set "-w 1" then the result does cycle between 100 and 109, but the solution does not scale.

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 6, 2019, 3:37am UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/9 "2019-06-06T03:37:26Z")

</div>

This might not be the best approach and i am trying to avoid an issues with InfluxDB mentioned [here](https://github.com/logstash-plugins/logstash-output-influxdb/issues/69). The only work arounds are to use uuid's as another field or to modify values for fields so that the timestamp values are not similar for at least 5-10 sec.  
I thought running through a loop an adding it to timestamps is the best way.

Do you think if writing a shell script to update timestamp values for every record would help ?

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 6, 2019, 2:09pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/10 "2019-06-06T14:09:26Z")

</div>

Is it possible to update the events before they reach filters ? someway after reading the messages through input plugins and before filters ?

---

<div class="post-metadata">

**Author:** ![naveenrt23](https://avatars.discourse-cdn.com/v4/letter/n/47e85d/32.png) [@naveenrt23](https://discuss.elastic.co/u/naveenrt23)\
**Post date:** [June 6, 2019, 3:03pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/11 "2019-06-06T15:03:03Z")

</div>

> [@Badger](#):
>
> ruby { init =\> ' @c = 0 ' code =\> ' event.set("timex", @c + event.get("time")) @c += 1 if @c \>= 10 @c = 0 end ' } }

I tried something similar to this but I get "Ruby exception occurred: undefined local variable or method `count' for #LogStash::Filters::Ruby:0x138f09da" error.

code:

```
ruby {
    init => "count = 0"
    code => "event.set('timestamp', event.get('timestamp_object').to_i * 1000);
    event.set('time',event.get('timestamp_object').to_i*100000000);
        if count<=10
        count=count+1
        puts count
        event.set('time',event.get('time')+count)
        puts time
        end
        if count==10
        count=0
        puts 'resetting count to 0'
        end"
  }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 4, 2019, 3:03pm UTC](https://discuss.elastic.co/t/how-to-write-while-loops-inside-plugins-or-filters/184426/12 "2019-07-04T15:03:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.