# Trouble with conditionals in aggregate filter **URL:** \ **Category:** Logstash\ **Created:** [July 23, 2019, 8:35pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910 "2019-07-23T20:35:55Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![N\_Antech](https://avatars.discourse-cdn.com/v4/letter/n/898d66/32.png) [@N\_Antech](https://discuss.elastic.co/u/N_Antech)\ **Post date:** [July 23, 2019, 8:35pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/1 "2019-07-23T20:35:55Z")
I am working to pull my data into elasticsearch using logstash and running into some trouble. I'll do my best to explain the situation: I'm pulling data via jdbc and everything seems to be working with the exception of one issue. my data looks something similar to: id, name, test\_name, test\_score, question\_name, question\_score ``` map['tests'] ||= [] map['tests'] << {event.get('test_name') => event.get('test_score')} map['assays'] ||= [] map['assays'] << {event.get('question_name') => event.get('question_score')} ``` Now, as expected, if I have a test name of "Final" with multiple questions "Bonus", "Essay" etc I will have multiple entries in the output for tests. I am trying to use a conditional to filter this but if I do: `if !map['tests'][event.get('test_name')]` I get No implicit conversion of string into integer if I do: `if event.get('test_name') not in map['tests']` I get unexpected keyword\_not, if I reverse the logic I get unexpected keyword\_in. I found a page on SO mentioning using something more similar to `if ! map['tests'][0][event.get('test_name')]` This results in undefined method `'[]'` for nil:NilClass I'm really unsure of where to go from here. If anyone could point me in the right direction I'd greatly appreciate it! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 23, 2019, 9:45pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/2 "2019-07-23T21:45:45Z")
> [@N\_Antech](#): > > I am trying to use a conditional to filter this but if I do: > `if !map['tests'][event.get('test_name')]` I do not understand what you are trying to test here. map['tests'] is an array of hashes. Are you trying to test whether any of the hashes in the array have a key equal to the value of the [test\_name] field. ---
**Author:** ![N\_Antech](https://avatars.discourse-cdn.com/v4/letter/n/898d66/32.png) [@N\_Antech](https://discuss.elastic.co/u/N_Antech)\ **Post date:** [July 23, 2019, 10:02pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/3 "2019-07-23T22:02:44Z")
that's exactly it, There might be multiple fields that end up as map['tests']['Final'] for example, they will be exactly the same so I'm hoping to filter them out. For example I might currently end up with something akin to: ``` tests{ "Final":86 "Final":86 "Final":86 "Midterm":70 } ``` I can't easily handle this at the SQL side because the second aggregated field ("assays" in the above example) will have multiple entries associated with a test. The essential layout in the database is that client has a one to many relationship with tests which then has a one to many relationship with assays. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 23, 2019, 10:08pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/4 "2019-07-23T22:08:22Z")
> [@N\_Antech](#): > > tests{ "Final":86 "Final":86 "Final":86 "Midterm":70 } That's not what you are building. You are building an array of hashes. ``` [{"Final":"86"}, {"Final":"86"}, {"Final":"86"}, {"Midterm":"90"}] ``` ---
**Author:** ![N\_Antech](https://avatars.discourse-cdn.com/v4/letter/n/898d66/32.png) [@N\_Antech](https://discuss.elastic.co/u/N_Antech)\ **Post date:** [July 23, 2019, 10:18pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/5 "2019-07-23T22:18:42Z")
That's fair, I'm not very familiar with what is happening. My example was more meant to be informal on what data exists and your example is more similar to what I see in kibana. My questions is, how would I avoid having 4 hashes in the array and only end with the two unique ones`[{"Final":"86"}, {"Midterm":"90"}]` ? Is this even possible? If there's a better way to import the data in a different format I'm open to it. I'm just getting started with elasticsearch and logstash (clearly) and don't currently have any hard constraints on how I organize my data since this project is currently only going to be used by me. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 23, 2019, 10:24pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/6 "2019-07-23T22:24:21Z")
Then I think you want the tests object to be a hash. ``` map["tests"] ||= {} map["tests"][event.get("test_name")] = event.get("test_score") ``` ---
**Author:** ![N\_Antech](https://avatars.discourse-cdn.com/v4/letter/n/898d66/32.png) [@N\_Antech](https://discuss.elastic.co/u/N_Antech)\ **Post date:** [July 23, 2019, 11:09pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/7 "2019-07-23T23:09:04Z")
This worked beautifully. Thank you so much for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 20, 2019, 11:16pm UTC](https://discuss.elastic.co/t/trouble-with-conditionals-in-aggregate-filter/191910/8 "2019-08-20T23:16:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.