Can someone please help with mapping when there are more than 1 nested type properties in the aggregate mapping
I am seeing a duplicate of data getting created in the document for the nested type properties, when in the Logstash Config file I am mapping more than 1 nested type properties.
below is my oracle table output.
i created a nested table as below
PUT test
{
"settings": {
"index.mapping.coerce": false
},
"mappings": {
"dynamic": "strict",
"properties" : {
"agreementId" : {
"type" : "text",
"copy_to" : [
"primaryFields"
]
},
"customers" : {
"properties" : {
"customerId" : {
"type" : "keyword",
"index" : false,
"doc_values" : false
},
"customerAddresses" : {
"type" : "nested",
"properties" : {
"custAddress" : {
"type" : "text"
},
"custAddressType" : {
"type" : "keyword",
"doc_values" : false
}
}
},
"phones" : {
"properties" : {
"phonenumber" : {
"type" : "text",
"copy_to" : [
"primaryFields"
]
},
"phonetype" : {
"type" : "keyword",
"doc_values" : false
}
}
}
}
}
}
}
}
when I tried to insert record via logstagh , I am getting duplicate value in phone number in address as below which i should get only one
{
"took": 474,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "test",
"_id": "123456789",
"_score": null,
"_source": {
"agreementId": 123456789,
"customers": [
{
"phones": [
{
"phonetype": "Cell",
"phonenumber": "2222222222"
},
{
"phonetype": "Cell",
"phonenumber": "2222222222"
},
{
"phonetype": "Home",
"phonenumber": "1111111111"
},
{
"phonetype": "Home",
"phonenumber": "1111111111"
}
],
"customerAddresses": [
{
"custAddressType": "Mailing",
"custAddress": "123 Main St."
},
{
"custAddressType": "Billing",
"custAddress": "456 South"
},
{
"custAddressType": "Mailing",
"custAddress": "123 Main St."
},
{
"custAddressType": "Billing",
"custAddress": "456 South"
}
]
}
]
},
"sort": [
1713679200000
]
}
]
}
}
I used aggregate and below is the mapping for address and phone number .
map['customers'].each { |cus|
if cus['customerId'] == event.get('customer') && event.get('Contact') != nil
cus['phones'] ||=[]
cus['phones'] << {
'phonenumber' => event.get('Contact'),
'phonetype' => event.get('Contacttype'),
}
map['customers'].each { |cus|
if cus['customerId'] == event.get('customer_id') && event.get('Address') != nil
cus['customerAddresses'] ||=[]
cus['customerAddresses'] << {
'custAddress' => event.get('Address'),
'custAddressType' => event.get('Addresstype'),
}
How to avoid duplicate record insert in the nested document.