# Elasticsearch-jdbc plugin fetching only part of data when nesting documents

**URL:** https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600
**Category:** Elasticsearch
**Created:** [May 13, 2015, 7:58am UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600 "2015-05-13T07:58:33Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![askawinska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/askawinska/32/44860_2.png) [@askawinska](https://discuss.elastic.co/u/askawinska)
#### Post date: [May 13, 2015, 7:58am UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600/1 "2015-05-13T07:58:33Z")

</div>

Hi everyone,

I have successfully employed elasticsearch-jdbc plugin to pump my data (in river mode) from a Postgres db. Everything ran smoothly until I tried to map my data from joined tables into nested documents - I found the river to be fetching only a part of the embedded objects.

The SQL query, run independently from my postgres console, yields 15 rows. In ES index there seem to be only 6 of them merged into nested documents. (even though ES console claims to have imported 15 rows at a time)

Have I messed up my sql query or is it some bug in elasticsearch-jdbc merging the documents?

Cheers,

Anna

- river definition:

- my\_river.json:

- SQL query results:

\_id; tag[id\_type]

```
1;"2c"
1;"4c"
1;"22a"
1;"1a"
2;"3c"
3;"3c"
1;"2a"
2;"1c"
3;"1c"
4;"4c"
2;"2c"
2;"4c"
3;"4c"
1;"3c"
1;"1c"

```

- elasticsearch query result:

- Elasticsearch console:

---

<div class="post-metadata">

### Author: ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [May 13, 2015, 9:08am UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600/2 "2015-05-13T09:08:43Z")

</div>

In the rows from the SQL result set, the ids are not monotonic.

For example, the doc id 1 appears three times, at the beginning (with "2c","4c","22a","1a"), in the middle (with "2a"), and at the end (with "3c", "2c").

The order of rows (the doc ids) is important for correct JSON doc construction. The indexing process is not smart and can not reorder the rows for you. For this, you must use SQL `ORDER BY` clause.

---

<div class="post-metadata">

### Author: ![askawinska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/askawinska/32/44860_2.png) [@askawinska](https://discuss.elastic.co/u/askawinska)
#### Post date: [May 13, 2015, 11:02am UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600/3 "2015-05-13T11:02:06Z")

</div>

Many thanks! Worked like a charm.  
But then, I assume, updating the nested documents in case the join results change, won't work that easily either?

For documentational purposes, what has helped was:

```
select * from (
SELECT
      v1.id as _id,
      c.id || 'c' as "tag[id_type]"
      FROM exit e
      INNER JOIN
      venue v1 ON e.id = v1.exit_id
      INNER JOIN category_venue cv ON v1.id = cv.venue_id
      INNER JOIN category c on c.id = cv.category_id
      UNION SELECT
      v2.id as _id, a.id || 'a' as "tag[id_type]"
      FROM exit e
      INNER JOIN venue v2 ON e.id = v2.exit_id
      INNER JOIN amenity a ON v2.id = a.venue_id
      ) result
      order by result._id
```

---

<div class="post-metadata">

### Author: ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [May 13, 2015, 12:07pm UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600/4 "2015-05-13T12:07:47Z")

</div>

Each doc id is indexed as a new document. Updating a doc by id is not implemented.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:14am UTC](https://discuss.elastic.co/t/elasticsearch-jdbc-plugin-fetching-only-part-of-data-when-nesting-documents/600/5 "2017-07-06T00:14:18Z")

</div>


