hi @craigothy, can you provide us with the entire azure configuration (obviously replace any sensitive information)?
also if you try using the azure eventhub namespace's connection string using the secondary key instead (in case you used the primary one) are you encountering the same issue?
It looks like the input string we're trying to decode is is not in Base64 encoded form.
Mariana, thank you. I am still getting the same error with the secondary key. It is almost as if there is a parsing issue? Here is an excerpt of my config:
The namespace config option is removed and we are introducing storage_account: the name of the storage account the state/offsets will be stored and updated, and storage_account_key
: the storage account key, this key will be used to authorize access to data in your storage account.
These new settings are required.
I am currently investigating this scenario, I have tested on 7.6.0 and 7.6.1 I get:
2020-03-31T17:14:21.865+0200 ERROR [azure-eventhub input] azureeventhub/input.go:116 illegal base64 data at input byte 0 {"connection string": "<no value>"}
because I have not disabled the other filesets.
But the enabled one seems to work fine.
Thanks Mariana. I am no longer getting that error based on the following config. Although I am not seeing any azure events collected. Can you tell me what might be the best way to troubleshoot that? Thank you!
Great, I will try that. I see references to pulling in Azure data using logstash or filebeat. From Elastic's perspective do you have any insight into what is the best method going forward? Thank you, Mariana.
Depends on your use case, you can use filebeat if you just need a lightweight shipper for your logs, if you need to do any extra transformation of this data before it hits your es cluster you might want to have a look at logstash.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.