# Avoid duplicate in nested table via logstagh

**URL:** <https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123>\
**Category:** Logstash\
**Tags:** elastic-stack-sql\
**Created:** [May 9, 2024, 4:24am UTC](https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123 "2024-05-09T04:24:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![karthikeyanc2003](https://avatars.discourse-cdn.com/v4/letter/k/ea5d25/32.png) [@karthikeyanc2003](https://discuss.elastic.co/u/karthikeyanc2003)\
**Post date:** [May 9, 2024, 4:24am UTC](https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123/1 "2024-05-09T04:24:53Z")

</div>

Can someone please help with mapping when there are more than 1 nested type properties in the aggregate mapping

I am seeing a duplicate of data getting created in the document for the nested type properties, when in the Logstash Config file I am mapping more than 1 nested type properties.  
below is my oracle table output.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f1ed7e02781c874b8ecc4c732a87994f5137bfdb.png)

i created a nested table as below

* * *

```auto
PUT test
{
  "settings": {
    "index.mapping.coerce": false
  },
  "mappings": {
    "dynamic": "strict",
    "properties" : {
    "agreementId" : {
          "type" : "text",
          "copy_to" : [
            "primaryFields"
          ]
        },
        "customers" : {
          "properties" : {
            "customerId" : {
              "type" : "keyword",
              "index" : false,
              "doc_values" : false
            },
        "customerAddresses" : {
              "type" : "nested",
              "properties" : {
                "custAddress" : {
                  "type" : "text"
                },
                "custAddressType" : {
                  "type" : "keyword",
                  "doc_values" : false
                }
              }
            },
        "phones" : {
              "properties" : {
                "phonenumber" : {
                  "type" : "text",
                  "copy_to" : [
                    "primaryFields"
                  ]
                },
                "phonetype" : {
                  "type" : "keyword",
                  "doc_values" : false
                }
              }
            }
          }
        }
    }
  }
}

```

* * *

when I tried to insert record via logstagh , I am getting duplicate value in phone number in address as below which i should get only one

* * *

```auto
{
    "took": 474,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "test",
                "_id": "123456789",
                "_score": null,
                "_source": {
                    "agreementId": 123456789,
                    "customers": [
                        {
                            "phones": [
                                {
                                    "phonetype": "Cell",
                                    "phonenumber": "2222222222"
                                },
                                {
                                    "phonetype": "Cell",
                                    "phonenumber": "2222222222"
                                },
                                {
                                    "phonetype": "Home",
                                    "phonenumber": "1111111111"
                                },
                                {
                                    "phonetype": "Home",
                                    "phonenumber": "1111111111"
                                }
                            ],
                            "customerAddresses": [
                                {
                                    "custAddressType": "Mailing",
                                    "custAddress": "123 Main St."
                                },
                                {
                                    "custAddressType": "Billing",
                                    "custAddress": "456 South"
                                },
                                {
                                    "custAddressType": "Mailing",
                                    "custAddress": "123 Main St."
                                },
                                {
                                    "custAddressType": "Billing",
                                    "custAddress": "456 South"
                                }
                            ]
                        }
                    ]
                },
                "sort": [
                    1713679200000
                ]
            }
        ]
    }
}

```

* * *

I used aggregate and below is the mapping for address and phone number .

```auto
map['customers'].each { |cus|
                                        if cus['customerId'] == event.get('customer') && event.get('Contact') != nil
                                                cus['phones'] ||=[]
                                                cus['phones'] << {
                                                'phonenumber' => event.get('Contact'),
                                                'phonetype' => event.get('Contacttype'),
                                                }

```

```auto
    map['customers'].each { |cus|
                                        if cus['customerId'] == event.get('customer_id') && event.get('Address') != nil
                                                cus['customerAddresses'] ||=[]
                                                cus['customerAddresses'] << {
                                                'custAddress' => event.get('Address'),
                                                'custAddressType' => event.get('Addresstype'),
                                                }

```

How to avoid duplicate record insert in the nested document.

---

<div class="post-metadata">

**Author:** ![karthikeyanc2003](https://avatars.discourse-cdn.com/v4/letter/k/ea5d25/32.png) [@karthikeyanc2003](https://discuss.elastic.co/u/karthikeyanc2003)\
**Post date:** [May 9, 2024, 4:26am UTC](https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123/2 "2024-05-09T04:26:09Z")

</div>

I saw few post related to the same but non of them are answered and closed without any answer . Need your help on the same