# Question about nested data

**URL:** <https://discuss.elastic.co/t/question-about-nested-data/239829>\
**Category:** Logstash\
**Created:** [July 3, 2020, 3:23pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829 "2020-07-03T15:23:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mirakuru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirakuru/32/71602_2.png) [@Mirakuru](https://discuss.elastic.co/u/Mirakuru)\
**Post date:** [July 3, 2020, 3:23pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/1 "2020-07-03T15:23:16Z")

</div>

Im trying to read a 100k line file into elastic search with logstash.

I have a .txt file lines are like this userid;action.

I want to use my userids as uniq and import every matching action to it, but there are duplicated actions which i dont want..

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 3, 2020, 3:37pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/2 "2020-07-03T15:37:32Z")

</div>

Set the document\_id based on the userid (either as-is, or using a fingerprint filter) and then duplicate actions for the same userid will be overwritten.

---

<div class="post-metadata">

**Author:** ![Mirakuru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirakuru/32/71602_2.png) [@Mirakuru](https://discuss.elastic.co/u/Mirakuru)\
**Post date:** [July 3, 2020, 3:42pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/3 "2020-07-03T15:42:36Z")

</div>

yes but i also want to keep new actions nested to userid too, with what you are saying it would overwrite action wouldnt it?

what i need is:

add  
userid = 3  
action = walk

if new data matches userid3 and the new action is uniq add it to action list etc  
action = walk, talk

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 3, 2020, 3:47pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/4 "2020-07-03T15:47:35Z")

</div>

In that case use an aggregate filter to collect all the unique actions for a userid (using a Ruby [set](https://ruby-doc.org/stdlib-2.7.1/libdoc/set/rdoc/Set.html), perhaps). If the input is not sorted it would be like [example 3](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example3), if it is sorted by userid then example 4. Note that this means logstash will be holding the entire input file in memory until the timeout triggers.

---

<div class="post-metadata">

**Author:** ![Mirakuru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mirakuru/32/71602_2.png) [@Mirakuru](https://discuss.elastic.co/u/Mirakuru)\
**Post date:** [July 3, 2020, 3:48pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/5 "2020-07-03T15:48:45Z")

</div>

would using php with elasticsearch module to do this without using logstash would perform good?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 3, 2020, 4:00pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/6 "2020-07-03T16:00:17Z")

</div>

I could not say.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2020, 4:03pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829/7 "2020-07-31T16:03:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.