Currently i have an ingest node configured to put documents in a separate index in the event of a failure as such:
"on_failure" : [
{
"set" : {
"field" : "_index",
"value" : "failed-{{ _index }}"
}
}
]
However, the issue is that it is difficult for me to find what is causing the failure when a document is placed in the "failure index".
Is there a way to enable logging to see what went wrong? Are there better ways of identifying the issues/problems?