Failed to execute pipeline PIPELINE DOES NOT EXIST

Elastic Stack Elasticsearch
1

Elastisearch Version: elasticsearch-5.5.0-1.noarch
Filebeat Version: filebeat-5.5.0-1.x86_64

Can anyone please enlighten me on why I am getting the pipeline does not exist error, but when I check it's there? I also used the Kibana Dev UI to verify and it's there also, and curl against every elasticsearch node shows it's there.

Error from filebeat log:
2017-07-10T15:35:31Z WARN Can not index event (status=400): {"type":"illegal_argument_exception","reason":"pipeline with id [filebeat-5.5.0-haproxy-haproxy_log-pipeline] does not exist"}

Error from elasticsearch log:
[2017-07-10T15:39:36,926][DEBUG][o.e.a.b.TransportBulkAction] [es1.ops.us-east-1] failed to execute pipeline [filebeat-5.5.0-haproxy-haproxy
_log-pipeline] for document [filebeat-2017.07.10/doc/null]
java.lang.IllegalArgumentException: pipeline with id [filebeat-5.5.0-haproxy-haproxy_log-pipeline] does not exist
at org.elasticsearch.ingest.PipelineExecutionService.getPipeline(PipelineExecutionService.java:194) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.ingest.PipelineExecutionService.access$100(PipelineExecutionService.java:41) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.ingest.PipelineExecutionService$2.doRun(PipelineExecutionService.java:88) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.5.0.jar:5.5.0
]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.5.0.jar:5.5.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

I have run a curl command to verify this pipeline DOES exist on all elasticsearch nodes and it does.

Actually all of my pipelines are returning the same error. Very strange.

Also my elasticsearch cluster is configured as follows on all nodes:

discovery.zen.ping.unicast.hosts: es1,es2,es3,es4,es5,es6,es7,es8,es9,es10,es11
discovery.zen.minimum_master_nodes: 3
node.ingest: true

2

Update on this.

I manually deleted ALL pipelines from previous version of filebeat, and xpack which appears to have been the actual problem.

Now pipelines are being located properly.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.