Hi all,
We are experiencing very strange behaviour from the Azureblob plugin. I am not sure if it's an issue with the plugin, or with Logstash.
The data is being parsed correctly, but Logstash seems to lag behind quite a bit. The data we see coming in Kibana seems to have a delay of several hours (+-2,5 hours on 1 environment, 1 whole week for another environment).
This delay seems to be relatively stable per environment. In other words: It's always roughly 2,5 hours behind.
Is there any mechanism that manages where the plugin is supposed to "resume" reading the data? Could this be giving issues? Any pointers are welcome!
The config is as follows:
input {
azureblob
{
storage_account_name => "correct_name
storage_access_key => "very_long_key"
container => "container_name"
codec => "json"
type => "typename"
}
}
Thanks in advance!