# How to load nested documents in Elasticsearh using logstash

**URL:** https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501
**Category:** Logstash
**Created:** [December 19, 2017, 8:24pm UTC](https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501 "2017-12-19T20:24:12Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Geert](https://avatars.discourse-cdn.com/v4/letter/g/848f3c/32.png) [@Geert](https://discuss.elastic.co/u/Geert)
#### Post date: [December 19, 2017, 8:24pm UTC](https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501/1 "2017-12-19T20:24:12Z")

</div>

I have a product database with 100.000 products. Each product has its own specs. The number and type of specs differ for each product. So my database has 2 tables with a 1:N relation: a product table (about 100.000 records) and a spec table (about 600.000 records). The product-ID is the key for the product table and product-ID + sequenceNo is the key for the spec. I want to index this database in Elasticsearch as one nested document, using logstash. Can someone tell me how to do this.

Structure of my document index:  
{  
"mappings": {  
"products": {  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"@version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"productid": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 20  
}  
}  
},  
"description": {  
"type": "text",  
"analyzer": "dutch",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 120  
}  
}  
},  
"specs": {  
"type": "nested",  
"properties": {  
"productid": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 20  
}  
}  
},  
"seqno": {  
"type": "long"  
},  
"featurename": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 60  
}  
}  
},  
"featurevalue": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 60  
}  
}  
}  
}  
}  
}  
}  
}  
}

My Logstash configuration file looks like this:

input {  
jdbc {  
jdbc\_driver\_library =\> "mysql-connector-java-5.1.42-bin.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
jdbc\_connection\_string =\> "jdbc:mysql://......"  
jdbc\_user =\> "....."  
jdbc\_password =\> "........"  
jdbc\_validate\_connection =\> true  
schedule =\> "\* \* \* \* \*"  
statement =\> "select \* from webproducts"  
type =\> "products"  
}

jdbc {  
jdbc\_driver\_library =\> "mysql-connector-java-5.1.42-bin.jar"  
jdbc\_driver\_class =\> "com.mysql.jdbc.Driver"  
jdbc\_connection\_string =\> "jdbc:mysql://......"  
jdbc\_user =\> "....."  
jdbc\_password =\> "........"  
jdbc\_validate\_connection =\> true  
schedule =\> "\* \* \* \* \*"  
statement =\> "select \* from webspecs"  
type =\> "specs"  
}  
}

filter {  
mutate {  
add\_field =\> { "[@metadata][type]" =\> "%{type}" }  
remove\_field =\> ["type"]  
}  
}

output {  
if [@metadata][type] == "products" {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> ......  
password =\> ......  
index =\> .........  
document\_type =\> "%{[@metadata][type]}"  
document\_id =\> "%{productid}"  
}  
}

if [@metadata][type] == "specs" {  
elasticsearch {  
hosts =\> ["localhost:9200"]  
user =\> ......  
password =\> ......  
index =\> .........  
document\_type =\> "%{[@metadata][type]}"  
document\_id =\> "%{sku}.%{seqno}"  
}  
}  
}

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 16, 2018, 8:24pm UTC](https://discuss.elastic.co/t/how-to-load-nested-documents-in-elasticsearh-using-logstash/112501/2 "2018-01-16T20:24:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.