Filebeat Azure Module Pipeline Functions

[craigothy]

Could someone tell me what the difference is between the different azure module pipelines are? For example, I see:
filebeat-7.6.2-azure-activitylogs-pipleline
AND
filebeat-7.6.2-azure-activitylogs-azure-shared-pipeline

Not sure of what the use cases are for each of these? Thanks!

[shaunak (Shaunak Kashyap)]

Hi @craigothy,

The azure Filebeat module consists of 3 filesets: activitylogs, auditlogs, and signinlogs. Each of them define their own ingest pipelines, e.g. filebeat-7.6.2-azure-activitylogs-pipeline, to perform processing specific to that fileset. However, internally, each of them also need to do some common processing that's not specific to that fileset. This common processing is defined in a shared pipeline. Because of the way Filebeat is currently implemented, you get 3 copies of the shared pipeline, one for each fileset, but their contents should all be the same.

In short, you can ignore the *-azure-shared-pipeline pipelines. They are for internal use by the other azure ingest pipelines.

Hope that helps,

Shaunak