# Problem Sorting Unique Values in CSV

**URL:** <https://discuss.elastic.co/t/problem-sorting-unique-values-in-csv/215057>\
**Category:** Logstash\
**Created:** [January 14, 2020, 10:47pm UTC](https://discuss.elastic.co/t/problem-sorting-unique-values-in-csv/215057 "2020-01-14T22:47:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![niall](https://avatars.discourse-cdn.com/v4/letter/n/73ab20/32.png) [@niall](https://discuss.elastic.co/u/niall)\
**Post date:** [January 14, 2020, 10:47pm UTC](https://discuss.elastic.co/t/problem-sorting-unique-values-in-csv/215057/1 "2020-01-14T22:47:12Z")

</div>

Hi,

I've been trying to solve this issue and came to a dead end. I will need some advice for this.

I do have a CSV file. Below is just sample data, but the file has like up to thousand lines. Below is the sample file:

```auto
"timestamp","name","type","value","date"
"2019-11-27 05:42:45.000","bill.acme.com","ns","ns-1468.awsdns-55.org","201911"
"2019-11-27 05:42:45.000","bill.acme.com","ns","ns-1777.awsdns-30.co.uk","201911"
"2019-11-27 05:42:45.000","bill.acme.com","ns","ns-258.awsdns-32.com","201911"
"2019-11-27 05:42:45.000","bill.acme.com","ns","ns-638.awsdns-15.net","201911"
"2019-11-27 05:51:28.000","bills-record.global.acme.com","a","10.144.152.62","201911"
"2019-11-27 05:51:28.000","bills-record.global.acme.com","a","10.144.126.177","201911"
"2019-11-27 05:51:28.000","bills-record.global.acme.com","a","10.144.135.205","201911"

```

What I'm trying to achieve here is, if `name`,`type` and `date` has the same value, the output should be a single name value with multiple IP addresses/string as it's value. However the value of IP/string should not be a duplicate. The values can amount from 1 to n.

I've tried using IF statements but to no avail. I know by using `add_field`, it will automatically create an array if the field is not, but I can't get it to grab multiple IP addresses.

The sample JSON output I'm hoping is as below:

```auto
{
          "type" => "ns",
          "date" => "201911",
    	  "name" => "bill.acme.com",
      "@version" => "1",
     "timestamp" => "2019-11-27 05:42:45.000",
    	 "value" => [
        [0] "ns-1468.awsdns-55.org",
        [1] "ns-1777.awsdns-30.co.uk",
        [2] "ns-258.awsdns-32.com",
        [3] "ns-638.awsdns-15.net"
    ]
}
{
          "type" => "a",
          "date" => "201911",
    	  "name" => "bills-record.global.acme.com",
      "@version" => "1",
     "timestamp" => "2019-11-27 05:51:28.000",
    	 "value" => [
        [0] "10.144.152.62",
        [1] "10.144.126.177",
        [2] "10.144.135.205"
    ]
}

```

Thanks! 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 14, 2020, 11:10pm UTC](https://discuss.elastic.co/t/problem-sorting-unique-values-in-csv/215057/2 "2020-01-14T23:10:28Z")

</div>

You can do it using an [aggregate](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example4) filter. Make sure you have --pipeline.workers set to 1 and that you have java execution turned off (either on the command line or with 'java.pipeline\_execution: false' in logstash.yml

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 11, 2020, 11:10pm UTC](https://discuss.elastic.co/t/problem-sorting-unique-values-in-csv/215057/3 "2020-02-11T23:10:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.