# What configuration is used to add newly added data in old indexes? **URL:** \ **Category:** Logstash\ **Created:** [July 25, 2019, 9:39am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212 "2019-07-25T09:39:45Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![pyerunka](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Post date:** [July 25, 2019, 9:39am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/1 "2019-07-25T09:39:45Z")
Hello, Can anyone help me in this? I have one index made up of database table which is created by using sql query. I want to add newly added data into it. I do not want to index whole data again. For example: database index has 5 documents. 2 new rows are added into my table from which i have created index. so i want to add that 2 rows in my index which has 5 documents. What setting i have to do in ES? Regards, Priyanka ---
**Author:** ![pyerunka](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Post date:** [July 29, 2019, 8:27am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/2 "2019-07-29T08:27:57Z")
Can anyone help me? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [July 29, 2019, 11:05am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/3 "2019-07-29T11:05:20Z")
Hi Priyanka, This is not so much a question about elasticsearch but more about whichever system you use to feed it. Logstash is one example of an ETL tool and [here](https://discuss.elastic.co/t/loading-incremental-data-into-elasticsearch-from-oracle-database/102007/3) is an example question about using it to do incremental updates from a database. If you have further questions it's probably best to ask in the Logstash forum ---
**Author:** ![pyerunka](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Post date:** [July 30, 2019, 9:40am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/4 "2019-07-30T09:40:09Z")
Hello, @Mark_Harwood thanks for reply!! i have marked this question to logstash forum. can anyone help me to solve following error? i am using sql\_last \_value in my logstash config file and try to load incremental data into Elasticsearch from oracle database. My logstash config file : > ``` > input { > jdbc { > jdbc_driver_library => "abcd" > jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver" > jdbc_connection_string => "abcde" > jdbc_user => "test" > jdbc_password => "xyz" > jdbc_fetch_size => "10000" > statement => "SELECT * FROM table_name WHERE LASTMODIFIEDDATE > :sql_last_value" > last_run_metadata_path => "E:/usr/share/logstash/last_run_metadata/.query.sql" > use_column_value => true > tracking_column => "LASTMODIFIEDDATE" > tracking_column_type =>"timestamp" > } > } > output{ > elasticsearch { > hosts => "localhost:9200" > index => "types" > } > } > > ``` and when i am running this i am getting below error: > Error: no implicit conversion of NilClass into String > Exception: TypeError how to resolve it? Regards, Priyanka ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 30, 2019, 12:52pm UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/5 "2019-07-30T12:52:14Z")
Is there are stack trace for that error message? Does "E:/usr/share/logstash/last\_run\_metadata/.query.sql" exist? If so, what does it contain? ---
**Author:** ![pyerunka](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Post date:** [July 31, 2019, 2:58am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/6 "2019-07-31T02:58:38Z")
Hello @Badger, Yes, '.query.sql' file exist at location E:/usr/share/logstash/last\_run\_metadata/.query.sql . And it has timestamp of when logstash config file last run. for example, --- 2019-07-30 09:13:30.794000000 Z. Regards, Priyanka ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 31, 2019, 1:30pm UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/7 "2019-07-31T13:30:52Z")
I would try moving that file aside so that it creates the file again. Again, is there a stack trace associated with that error? ---
**Author:** ![pyerunka](https://avatars.discourse-cdn.com/v4/letter/p/d6d6ee/32.png) [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Post date:** [August 2, 2019, 4:52am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/8 "2019-08-02T04:52:27Z")
Hi @Badger, No there is not any stack trace associated with that error. Regards, Priyanka ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 30, 2019, 5:03am UTC](https://discuss.elastic.co/t/what-configuration-is-used-to-add-newly-added-data-in-old-indexes/192212/9 "2019-08-30T05:03:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.