My custom module ignores pipeline

Elastic Stack Beats
1

Hi,

I builded a custom module for my application following this guide.

After the setup of this module i can see the messages form my module in kabana, but they are not parsed with my pipeline.

Below the steps that i did:

  1. setup filebeat .\filebeat.exe setup -e

  2. enable the module .\filebeat.exe modules enable <mymodule>

  3. setup the module .\filebeat.exe setup --modules <mymodule> -e

Note, I tested my pipeline using API.

There are others steps to do?

Thanks,
Gaetano

UPDATE:
If I run the below commands it seems work correctly:

  1. stop the filebeat service Stop-Service filebeat

  2. disable the module .\filebeat.exe modules disable <mymodule>

  3. run this command: .\filebeat.exe -modules=<mymodule> -setup -e

This behavior is really strange because the service is stopped, the module is disabled but in kibana i can see the messages correctly parsed.

2

When running filebeat setup, you should also pass --pipelines in order to install the ingest pipelines.

The later command not only setups, but also runs filebeat. This might have setup the module once the first event has been seen.

3

The pipelines are installed, i can see the ingest pipelines if i run this http request GET _ingest/pipeline/.

However, i tryed to run .\filebeat.exe setup --modules <mymodule> -e --pipelines but the parsing still doesn't work.

4

Now you got me confused. So you say the module is enabled, ingest node pipeline is correctly installed, but filebeat is not using the pipeline you did create?

Can you share your module definition, configuration and filebeat logs?

5

Yes.

You can see this commit.

In the logs i see only this two lines:

2018-12-06T17:20:21.172+0100 INFO instance/beat.go:608 Home path: [C:\temp\filebeat-6.4.3-windows-x86_64] Config path: [C:\temp\filebeat-6.4.3-windows-x86_64] Data path: [C:\temp\filebeat-6.4.3-windows-x86_64\data] Logs path: [C:\temp\filebeat-6.4.3-windows-x86_64\logs]
2018-12-06T17:20:21.212+0100 INFO instance/beat.go:615 Beat UUID: 0b0c66ab-f277-4a53-bac4-feb8806d976c
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.