# Remembering Field Values **URL:** \ **Category:** Logstash\ **Created:** [February 3, 2017, 2:56pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870 "2017-02-03T14:56:41Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 3, 2017, 2:56pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/1 "2017-02-03T14:56:41Z")
Hi, Does anyone have any solutions for how to store and re-use field values between events? For example if we have a log file that contains something like:- Counter 1 stuff.... stuff.... Counter 2 stuff... stuff... stuff... Counter 3 stuff... stuff... and so on. I need to be able to add the counter number which is currently active to the fields I'm grokking out of the "stuff..." lines. I believe that this is possible with the "aggregate" filter but I have never been able to get it to work, or seen an example of how it could be done. I know that there is a community plugin called memorize but I'd rather try to accomplish this with standard plugins, if possible. Thanks! ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 4, 2017, 4:26pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/2 "2017-02-04T16:26:10Z")
aggregate seems to be the right plugin for your need. If you look at logstash "master" documentation, you should find 4 examples explaining how you can use aggregate plugin. Have you seen it ? ---
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 4, 2017, 6:20pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/3 "2017-02-04T18:20:24Z")
@fbaligand Thanks, yes, I have seen it. But not been able to find a way to leverage those example to solve the -seemingly simple- problem outlined in my first post. Can you? ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 4, 2017, 6:55pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/4 "2017-02-04T18:55:55Z")
Like that, your need is not precise enough so that I can help you. Can you give some documents before aggregate plugin, and the expected result after aggregate plugin ? ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 4, 2017, 7:16pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/5 "2017-02-04T19:16:19Z")
After reading one more time your need, I think this kind of configuration could do the job : ``` if [type] == "Counter" { aggregate { task_id => "%{host}%{path}" code => "map['counter_id'] = event.get('counter_id')" } } if [type] != "Counter" { aggregate { task_id => "%{host}%{path}" code => "event.set('counter_id', map['counter_id'])" } } ``` ---
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 4, 2017, 8:43pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/6 "2017-02-04T20:43:30Z")
@fbaligand Thank you very much for your interest. I tried to implement your code but no joy just yet. Here is what I have; hopefully it will be obvious where I am going wrong? The (test) log file I am reading looks like this: ``` cycle 1 name: jellybeans some: chocolate cycle 2 name: bubblegum name: gumdrops name: candycanes cycle 3 name: sherbet name: candyfloss cycle 4 name: mintdrops name: icecream ``` My conf file is this: ``` input { file { path => "/Users/me/Downloads/logstash-2.3.1/source.log" ignore_older => 0 } } filter { if [message] =~ /.*cycle.*/ { grok { match => { "message" => ".*cycle\s(?\d*)"} } } if [cycle] { aggregate { task_id => "%{host}%{path}" code => "map['cycle'] = event.get('cycle')" } } else { aggregate { task_id => "%{host}%{path}" code => "event.set('cycle', map['cycle'])" } } } output { stdout { codec => rubydebug } } ``` Which I think is pretty much representative of the code/behaviour you suggested. I am not differentiating messages by [type] however as all my events are coming from the same log file so I cannot see where or why I would use different types.. I was thinking that I could just trigger the aggregate events based on what the event contains (a cycle value or not). The output I get is basically this: ``` Aggregate exception occurred. Error: undefined method `set' for # ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: candyfloss", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.648Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error} Aggregate exception occurred. Error: undefined method `get' for # ; Code: map['cycle'] = event.get('cycle') ; Map: {} ; EventData: {"message"=>"cycle 4", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.648Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP", "cycle"=>"4"} {:level=>:error} Aggregate exception occurred. Error: undefined method `set' for # ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: mintdrops", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.649Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error} Aggregate exception occurred. Error: undefined method `set' for # ; Code: event.set('cycle', map['cycle']) ; Map: {} ; EventData: {"message"=>"name: icecream", "@version"=>"1", "@timestamp"=>"2017-02-04T20:32:10.649Z", "path"=>"/Users/me/Downloads/logstash-2.3.1/source.log", "host"=>"MBP"} {:level=>:error} { "message" => "cycle 1", "@version" => "1", "@timestamp" => "2017-02-04T20:32:10.645Z", "path" => "/Users/me/Downloads/logstash-2.3.1/source.log", "host" => "MBP", "cycle" => "1", "tags" => [ [0] "_aggregateexception" ] } { "message" => "name: jellybeans", "@version" => "1", "@timestamp" => "2017-02-04T20:32:10.646Z", "path" => "/Users/me/Downloads/logstash-2.3.1/source.log", "host" => "MBP", "tags" => [ [0] "_aggregateexception" ] } { "message" => "name: chocolate", "@version" => "1", "@timestamp" => "2017-02-04T20:32:10.647Z", "path" => "/Users/me/Downloads/logstash-2.3.1/source.log", "host" => "MBP", "tags" => [ [0] "_aggregateexception" ] } ``` As I am sure you are aware I am looking for the current value of cycle to be present in all events. Really appreciate this! ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 4, 2017, 9:13pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/7 "2017-02-04T21:13:25Z")
Ok, I saw in your error stack trace that you use logstash 2.3.1. The code that I provided requires logstash 2.4 or more. With logstash 2.3.1, the right code are : `code => "map['cycle'] = event['cycle']"` And then : `code => "event['cycle'] = map['cycle']"` ---
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 4, 2017, 9:28pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/8 "2017-02-04T21:28:39Z")
Brilliant. Success! Thank you very much, this is great news... I've needed this functionality for a long time. Thanks again 😀 ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 4, 2017, 9:35pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/9 "2017-02-04T21:35:26Z")
Happy to see I solved your problem ! ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 5, 2017, 5:02pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/10 "2017-02-05T17:02:41Z")
If your problem is solved, could you mark my solution comment as "solution" to close the topic ? ---
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 5, 2017, 5:59pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/11 "2017-02-05T17:59:36Z")
@fbaligand Thats odd. I did exactly that about 17 seconds after testing your solution. The topic IS tagged as having a solution in my view of this thread: ![](https://us1.discourse-cdn.com/elastic/original/2X/c/c675f1dda9aa833e6c088b56081e3633d1f275aa.png) Is there somewhere else | something else I should do? ---
**Author:** ![fbaligand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fbaligand/32/5657_2.png) [@fbaligand](https://discuss.elastic.co/u/fbaligand)\ **Post date:** [February 5, 2017, 6:02pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/12 "2017-02-05T18:02:54Z")
Sorry, I should buy some glasses 🙂 ---
**Author:** ![Kryten](https://avatars.discourse-cdn.com/v4/letter/k/58956e/32.png) [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Post date:** [February 5, 2017, 6:18pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/13 "2017-02-05T18:18:24Z")
No problem at all. I'm still smiling at FINALLY having a solution to this problem. THANKS AGAIN ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 5, 2017, 6:18pm UTC](https://discuss.elastic.co/t/remembering-field-values/73870/14 "2017-03-05T18:18:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.