Azure Module Error

Using version 7.6.1
I am getting an error when starting filebeat with the azure module enabled. Any ideas what could be causing this? Thank you.

2020-04-08T21:35:17.140-0500    INFO    [azure-eventhub input]  azureeventhub/input.go:110      azure-eventhub input worker has started.        {"connection string": "Endpoint=sb://;SharedAccessKeyName=siemRead;SharedAccessKey=mybase64hubkey"}
2020-04-08T21:35:25.119-0500    INFO    [azure-eventhub input]  azureeventhub/eph.go:57 handler id: "dd6c53e7-6838-4ad5-aa7e-3cdfc3a96c21" is running
        {"connection string": "Endpoint=sb://;SharedAccessKeyName=siemRead;SharedAccessKey=mybase64hubkey"}

    ______                 __  __  __      __
   / ____/   _____  ____  / /_/ / / /_  __/ /_  _____
  / __/ | | / / _ \/ __ \/ __/ /_/ / / / / __ \/ ___/
 / /___ | |/ /  __/ / / / /_/ __  / /_/ / /_/ (__  )
/_____/ |___/\___/_/ /_/\__/_/ /_/\__,_/_.___/____/

=> processing events, ctrl+c to exit
2020-04-08T21:35:25.925-0500    ERROR   [azure-eventhub input]  azureeventhub/input.go:116      ->, /go/src/
===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====
Description=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Time:2020-04-08T21:25:40.5976395Z, Details:
   AuthenticationErrorDetail: The MAC signature found in the HTTP request 'vM/30x3+MUzUsENlTjDGIWd9fKEW0wq1kg+lndUZSqQ=' is not the same as any computed signature. Server used following string to sign: 'GET

x-ms-date:Thu, 09 Apr 2020 02:35:25 GMT
   Code: AuthenticationFailed
   Authorization: REDACTED
   User-Agent: [Azure-Storage/0.7 (go1.13.8; linux)]
   X-Ms-Client-Request-Id: [b8f6bd7e-767f-4825-4487-75b49e28579c]
   X-Ms-Date: [Thu, 09 Apr 2020 02:35:25 GMT]
   X-Ms-Version: [2018-11-09]
   RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
   Content-Length: [753]
   Content-Type: [application/xml]
   Date: [Wed, 08 Apr 2020 21:25:39 GMT]
   Server: [Microsoft-HTTPAPI/2.0]
   X-Ms-Error-Code: [AuthenticationFailed]
   X-Ms-Request-Id: [5f20619e-c01e-00a4-53ec-0d31eb000000]

        {"connection string": "Endpoint=sb://;SharedAccessKeyName=siemRead;SharedAccessKey=mybase64hubkey"}
2020-04-08T21:35:25.925-0500    INFO    [azure-eventhub input]  azureeventhub/input.go:117      azure-eventhub input worker has stopped.        {"connection string": "Endpoint=sb://;SharedAccessKeyName=siemRead;SharedAccessKey=mybase64hubkey"}

Hey @craigothy,

This seems to be an authentication problem. How are you configuring the input?

I am starting to wonder if there might be an issue with * being hardcoded as the API endpoint. In this particular situation, the storage account and event hub exist on Azure's government cloud * Do you know if this might be hard coded within the module? For example, DNS lookups for: resolve just fine do not resolve

2020-04-09T09:57:59.444-0500    ERROR   [azure-eventhub input]  azureeventhub/input.go:116      ->, /go/src/
HTTP request failed

Get dial tcp: lookup siemeventsclassic.** on no such host
        {"connection string": "Endpoint=sb://;SharedAccessKeyName=siemRead;SharedAccessKey=mybase64accesskey"}

Hey @craigothy,

Yes, this may be the issue, I have found a place where Filebeat is pointing directly to the public cloud, here:

I have created an issue to investigate that:

Thanks for reporting!

Thank you very much, Jamie. So could I simply replace the string PublicCloud with USGovernmentCloud and recompile the module?

I haven't tried, but I'd say that this could be a workaround, yes. You will need to recompile metricbeat. You can use mage package command to compile and generate packages ready to be used (it takes some time).

@jsoriano, thank you. maybe this will turn into a separate post but if I just wanted to recompile that one azure module, what would be the best way? do I just run "make package" within that directory and copy those files into an existing filebeat install? Thanks for any guidance!

You cannot recompile this input alone, it is included in the filebeat binary. I would recommend to use make package because it generates packages ready to be installed. mage package must be executed from the directory of the beat you want to compile, x-pack/filebeat directory in your case. Once the command finishes you will find the built packages in build/distributions.

You will need docker and mage, you can read how to install mage here:

Thanks @jsoriano. So my mage package completes successfully but I am just left with directories for each packages but not the actual binary packages themselves. Am I missing something. For example, the deb package is actually a directory with .tmpl files. Thanks again for all your help!

@craigothy usually directories with the names of the packages can be found in build/package, actual packages are in build/distributions.

Thanks @jsoriano! Completely overlooked that folder. Appreciate your help!

1 Like

@craigothy did the change to USGovernmentCloud work?

Actually, I never tried that. I saw the pull request and just incorporated those changes and recompiled. That worked fine. Thanks @jsoriano.

1 Like

Oh, even better, thanks for testing!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.