# Jdbc\_streaming with nested elements **URL:** \ **Category:** Logstash\ **Created:** [March 13, 2019, 7:19pm UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208 "2019-03-13T19:19:13Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\ **Post date:** [March 13, 2019, 7:19pm UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/1 "2019-03-13T19:19:13Z")
Hi, I have 3 tables that has the described connections: A-\>B-\>C Basically i have FK from A to B and B to C. I'm trying to put all the data into a unique document with a list of B inside A and a list of C inside B. This is possible? I'm using jdbc\_streaming to put the B elements into the A, but how do i put the C elements inside the B?? here is an example what i doing: ```auto input { jdbc { jdbc_driver_library => "driver" jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver" jdbc_connection_string => "connectString" jdbc_user => "user" jdbc_password => "pwd" schedule => "* * * * *" statement => " SELECT FILE_ID, STATUS_CODE FROM TABLE_A WHERE DATE BETWEEN '2019-01-04-16.04.56' AND '2019-01-04-16.04.57'" } } filter { jdbc_streaming { jdbc_driver_library => "driver" jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver" jdbc_connection_string => "connectString" jdbc_user => "user" jdbc_password => "pwd" parameters => { "idfile" => "file_id" } statement => " SELECT FILE_PROCESS_ID, LINE_ID FROM TABLE_B WHERE FILE_ID = :idfile" target => "B" } } output { file { codec => "json_lines" path => "fileintegration.json" } } ``` Any ideas? ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [March 14, 2019, 4:22pm UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/2 "2019-03-14T16:22:18Z")
How static is table B and C? jdbc\_static allows you to pull B and C into a local DB (Apache Derby) then you can do a local join of B -\> C type lookup. ---
**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\ **Post date:** [March 14, 2019, 4:50pm UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/3 "2019-03-14T16:50:35Z")
Hi, Table B and C are not static at all. Both tables are very intensive with new data, specially C (errors). The use case is processing files (ETL) and register the file, its processes (lines) and errors ( encountered in line). Cheers, ---
**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\ **Post date:** [March 18, 2019, 4:20pm UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/4 "2019-03-18T16:20:41Z")
No one have any idea about this? ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [March 21, 2019, 9:40am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/5 "2019-03-21T09:40:10Z")
Before we look at the possibilities, is there likely to be a timing problem? Are the child records written to B and C before the parent write to A, or maybe all are in a transaction? ---
**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\ **Post date:** [March 21, 2019, 10:40am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/6 "2019-03-21T10:40:21Z")
No, there is no timing problem. The data in C exists only if there is some data in B. The data in B exists only if there is some data in A. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [March 21, 2019, 10:58am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/7 "2019-03-21T10:58:17Z")
Great. In this case I think you need two jdbc\_streaming filters and the jdbc input. The jdbc input collects new records from `A`, then the first jdbc\_streaming filter collects a matching record from `B` and the second jdbc\_streaming filter collects a matching record from `C`. ---
**Author:** ![bigster](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@bigster](https://discuss.elastic.co/u/bigster)\ **Post date:** [March 21, 2019, 11:33am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/8 "2019-03-21T11:33:03Z")
I did that, but how to i put the records from C inside the data of B? That's what i'm asking? Currently i have B and C data inside the A record, not C inside B that is inside A. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [March 21, 2019, 11:44am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/9 "2019-03-21T11:44:42Z")
Let me check. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [March 21, 2019, 11:47am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/10 "2019-03-21T11:47:39Z")
On the second jdbc\_filter use the `target` setting. Specify a nested field name using square bracket syntax. e.g. `"[B][C]"` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 18, 2019, 11:50am UTC](https://discuss.elastic.co/t/jdbc-streaming-with-nested-elements/172208/11 "2019-04-18T11:50:42Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.