I'm trying to sync all our dropbox document to elasticsearch. Total 599,000 documents.
This is a slow process. In 12 hours only 80,000 documents added. But so far so good but after some hours of sync the process crashes. After solving the problem and restarting the sync. All is starting again with the first document. Is there something like a checkpoint so after a restart the process starts where it crashes?
Hey! Incremental syncs should make future syncs faster, but they do require a successful full sync first. If you're running into issues syncing data with the Dropbox connector that seem like bugs, feel free to file an issue at GitHub - elastic/connectors: Official Elastic connectors for third-party data sources. If the permission setup is unclear, let us know. I'm happy to hear how we can improve!
Is there something like a checkpoint so after a restart the process starts where it crashes?
There are no checkpoints for full syncs. You need to sync data again.