Task: Pulling data from SQLServer and Pushing records to Elasticsearch. Achieving this through triggering the logstash cmd after certain upstream conditional triggers finished.
Planning to do this by cmd.exe in c#.net process. Any better way to achieve?
Scenario to handle:
Need to send email if data transfer completed successfully.
Need to send email if unsuccessful and also perform some event. Unsuccessful Condition: Could be anything like server not available/disk full.
Also can we capture the last record transferred to Elasticsearch in the same consecutive request in case of failure and trigger some event?
First two are V important.
Also facing issue when ES stopped "dead ES instance" in logs/command window output, but LS dont stop waits for ES? How to get this out/terminate when no response after say 5 attempts by LS to ES?