# Add nested field with logstash and JDBC

**URL:** https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585
**Category:** Logstash
**Created:** [August 22, 2018, 4:03pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585 "2018-08-22T16:03:45Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)
#### Post date: [August 22, 2018, 4:03pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/1 "2018-08-22T16:03:45Z")

</div>

Hi,  
I am using Logstash to update my Elasticsearch index. I have index called my\_files, and I want to update its records with "metadata" field. "metadata" should be a nested field that contains some other fields I want to get from mysql database  
I have the following config file

`  
input {  
jdbc {  
jdbc\_connection\_string =\> "jdbc:mysql://s.o.me.ip:port/db\_name"  
jdbc\_user =\> "usernem"  
jdbc\_password =\> "password"  
jdbc\_driver\_library =\> "./mysql-connector-java-8.0.12.jar"  
jdbc\_driver\_class =\> "com.mysql.cj.jdbc.Driver"  
statement =\> "select \* from mytable"  
use\_column\_value =\> true  
clean\_run =\> true  
tracking\_column =\> id  
schedule =\> "/1 \* \* \* \* \*"  
jdbc\_paging\_enabled =\> true  
jdbc\_page\_size =\> 200000  
}

}

output {  
elasticsearch {  
"hosts" =\> "[http://localhost:9200](http://localhost:9200)"  
"index" =\> "my\_files"  
"action" =\> "update"  
"document\_id" =\> "%{id}"  
"document\_type" =\> "file"  
}

```
stdout {
	codec => json_lines
}

```

}`

this config files adds the fields from the query directly in the document however I want to make a nested field "metadata" in my\_files index and add these fields there. I know that I can use mutate and create a field but I don't know how to do this in an easy way without having to add them one by one.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [August 22, 2018, 4:07pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/2 "2018-08-22T16:07:38Z")

</div>

What will the `metadata` value look like? Is it dynamic? Does it depend on some value of a field sort of like a SQL JOIN?

---

<div class="post-metadata">

### Author: ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)
#### Post date: [August 22, 2018, 4:10pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/3 "2018-08-22T16:10:11Z")

</div>

In mySQL I have a table "mytable' with 8 fields , 2 of them should not be inserted here (id, and another foreign key), 2 fields of type json, and 4 fields of type char.

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [August 22, 2018, 4:15pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/4 "2018-08-22T16:15:24Z")

</div>

You describe the input structure, I asked about the structure you want to add to the document after it is read from the DB.

---

<div class="post-metadata">

### Author: ![M.alsioufi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/m.alsioufi/32/52269_2.png) [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)
#### Post date: [August 22, 2018, 4:16pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/5 "2018-08-22T16:16:38Z")

</div>

I would like to have a field called "metadata" and inside this field to have the mentioned 6 fields

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [August 22, 2018, 4:21pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/6 "2018-08-22T16:21:00Z")

</div>

You can use [jdbc\_streaming filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-jdbc_streaming.html) for that, to do what we call enrichment.

The linked doc has quite a good example.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 19, 2018, 4:21pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585/7 "2018-09-19T16:21:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
