# Indexing structured documents using Logstash JDBC plugin

**URL:** https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192
**Category:** Logstash
**Created:** [January 14, 2016, 5:55am UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192 "2016-01-14T05:55:36Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![oshai](https://avatars.discourse-cdn.com/v4/letter/o/85e7bf/32.png) [@oshai](https://discuss.elastic.co/u/oshai)
#### Post date: [January 14, 2016, 5:55am UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192/1 "2016-01-14T05:55:36Z")

</div>

Hi,

I'm new to Logstash and i have a simple scenario that I'm trying to execute.

Using the Logstash JDBC plugin I'm fetching data from MySQL and Indexing it in ES.

Before using Logstash I've used the elasticsearch jdbc plugin(jprante) which allowed me to index structured data using "." and even json arrays using brackets("[]").

> **[jprante/elasticsearch-jdbc](https://github.com/jprante/elasticsearch-jdbc#bracket-notation-for-json-array-construction)**
>
> elasticsearch-jdbc - JDBC importer for Elasticsearch

I'm trying to do the same with the logstash jdbc plugin but its not working.(Its not supporting dots)

I wanted to know how can I achieve the same things that I've done with the ES jdbc plugin using the logstash plugin(i.e. structured json objects with arrays).

Thanks and best regards,  
Orel

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 15, 2016, 6:55am UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192/2 "2016-01-15T06:55:29Z")

</div>

Use `[field][subfield]` to denote nested fields. See [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references). If this doesn't help you have to supply more details.

---

<div class="post-metadata">

### Author: ![oshai](https://avatars.discourse-cdn.com/v4/letter/o/85e7bf/32.png) [@oshai](https://discuss.elastic.co/u/oshai)
#### Post date: [January 17, 2016, 7:19am UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192/3 "2016-01-17T07:19:54Z")

</div>

Hi,

Thanks for the quick response.

I'm not sure how can I use this to solve my issue.

Lets say that I have a query(over MySQL) that I'm using for my input that return the following:  
ID FIELD\_1 FIELD\_2  
1 1 1  
1 1 2  
2 1 1

And I'll like to format them to the following json:  
{  
doc:[  
{  
id:1,  
group:[{  
FIELD\_1 : 1 ,  
FIELD\_2 : 1  
},  
{  
FIELD\_1 : 1 ,  
FIELD\_2 : 2  
}  
]  
},  
{  
id:2,  
group:[{  
FIELD\_1 : 1 ,  
FIELD\_2 : 1  
}  
]  
}  
]  
}

I can achieve that using the elasticsearch-jdbc plugin by "calling" the FIELD\_1 name as group[FIELD\_1]. i.e.  
select .... FIELD\_1 as group[FIELD\_1] , FIELD\_2 as group[FIELD\_2].

But I'm not sure how can I use the reference that you sent to get the same. Can you please provide an example?

I hop that its clear enough 🙂

Thanks and best regards,  
Orel

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 17, 2016, 4:24pm UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192/4 "2016-01-17T16:24:27Z")

</div>

I don't think there's an easy way of accomplishing this with Logstash. I'd go for a custom script or at least use a custom script to help transform the data—you could still use Logstash for polling the database and shipping to Elasticsearch (or whatever outputs you use).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:15am UTC](https://discuss.elastic.co/t/indexing-structured-documents-using-logstash-jdbc-plugin/39192/5 "2017-07-06T05:15:27Z")

</div>


