Filebeat errors on setup - Apache ingest pipelines failing to load

So here is my problem it seems filbeats setup is failing for me. I get a bunch of empty error messages

**"Exiting: 1 error: Error setting up ML for apache_ecs: 17 errors: ; ; ; ; ; ; ; ; ; ; error while".**

I'm thinking that this is because the error structure used by filebeats is wrong.
The structure is defined here: beats/importer.go at v7.11.1 · elastic/beats · GitHub.

And, the errors get appeneded here:beats/importer.go at v7.11.1 · elastic/beats · GitHub.

The errors in the response looks more like: kibana/types.ts at v7.11.1 · elastic/kibana · GitHub.

There was a merge request a while ago that looks like it changed the error responses https://github.com/elastic/kibana/pull/74965.

Does anyone know if filebeats is representing errors incorrectly or is this just another kind of error it doesn't know about?

The errors are just datafeed and job conflicts which it looks like the code was trying to watch for.

Hi @QueenAmidala Welcome to the community.

Long story short, don't worry about that right now... it has to do with some legacy ml jobs / mapping and should not be an issue.

Curious what version you are running though?

Go ahead and run filebeat and use the Visualizations, Dashboards, Alerts actions as you see fit let us know if you have any issues.

Thanks for checking in with this.

This seems to also be a problem for these people Error trying to setup filebeat because the error response structure for a visualization is actually more like kibana/types.ts at v7.11.1 · elastic/kibana · GitHub when you run filebeats setup for the first time and the error message doesn't match with what is looked for in the code.

Is there no desire to fix this because it is legacy code? Can the machine learning option for filebeat setup at least be disabled by default?

I'm using version 7.11.1 of Elastic. So, like Elasticsearch 7.11.1, Kibana 7.11.1, and Filebeat 7.11.1. I'm using the container images.

Hi @QueenAmidala

Thank you for opening the issue, I will poke engineering a bit tomorrow if I get a chance. When I asked before (as I ran into myself) I was told this will be fixed in / mitigated in a future release and at this point it is does not indicate any actual issues.

Is there any functionality other than the error message that is broken for you? If so let us know, otherwise I would proceed with the stack.

If you are using 7.11.1 and enable apache and / or ngnix and then run setup the correct ML jobs will be loaded.

I just built a brand new stack ...

$ ./filebeat modules enable apache nginx 
$ Enabled apache
$ Enabled nginx

$ ./filebeat setup
......
Exiting: 3 errors: Error setting up ML for apache_ecs: 10 errors: ; ; ; ; ; ; ; ; ; ; Error setting up ML for apache_ecs: 10 errors: ; ; ; ; ; ; ; ; ; ; Error setting up ML for nginx_ecs: 10 errors: ; ; ; ; ; ; ; ; ; 

I get the same error however when I go into Kibana the correct ML jobs are loaded for apache and nginx, my understanding is that is what would be broken if it was an actual issue.

It is a conflict error about a resource already existing since Nginx and Apache use the same names for things. But it has other repercussions besides being an error message, for example the filebeat setup command fails with a failing exit code, and also the pipeline loading gets skipped because it happens after the error

It's also not very intuitive to see empty error messages instead of the actual errors.

Hi @QueenAmidala

Yeah agree needs to be fixed, interesting I was about to tell you that the pipelines are being added because the nginx pipeline is loaded but the apache pipelines are not loaded! That is not good.

Could you add to your issues that the apache ingest pipelines are failing to load. (Bad error messages are one thing but failing to load the pipeline is not good at all) I will pass on the issue to engineering.

Actually if you editing the Title to something like

Filebeat errors on setup - Apache ingest pipelines failing to load

That would probably get more attention.

Thanks again.

As a workaround in the meantime you can run the following to manually load the pipelines.

./filebeat setup --modules=apache --pipelines

Hi @QueenAmidala

Thanks again as you can see Engineering has picked up your Github issue.

Above I was hoping that you would update the Github Issue Title... to

Filebeat errors on setup - Apache ingest pipelines failing to load

Its a little more descriptive