How to avoid large index update

Hi All
We are trying to sync the relational data from mssql database to ES index. The structure of my sql data base is like this:

  • Teacher table - teacher_id , teacher_name
  • Course table- course_id , course_name, teacher_id
  • Doubts table- doubt_id , doubt, course id, teacher_id

We have mapped the index in a denormalized way to get the results quickly(ie instead of teacher_id we are keeping the teacher_name in the index). The current index structure as below :
doubt: '',
course_name: '',
teacher_name: ''

This index contains millions of documents. When the data in teacher or course master table in sql database changes, it is taking a long time to update the index.
Is there any better way to update the index quickly? Is it possible to split the data into different indices and get the required output by joining the different indices ?

Appreciate your valuable suggestions.

Are you using the _bulk API?

we tried _bulk api but it is time consuming

What do you mean by that?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.