Elastisearch Version: elasticsearch-5.5.0-1.noarch
Filebeat Version: filebeat-5.5.0-1.x86_64
Can anyone please enlighten me on why I am getting the pipeline does not exist error, but when I check it's there? I also used the Kibana Dev UI to verify and it's there also, and curl against every elasticsearch node shows it's there.
Error from filebeat log:
2017-07-10T15:35:31Z WARN Can not index event (status=400): {"type":"illegal_argument_exception","reason":"pipeline with id [filebeat-5.5.0-haproxy-haproxy_log-pipeline] does not exist"}
Error from elasticsearch log:
[2017-07-10T15:39:36,926][DEBUG][o.e.a.b.TransportBulkAction] [es1.ops.us-east-1] failed to execute pipeline [filebeat-5.5.0-haproxy-haproxy
_log-pipeline] for document [filebeat-2017.07.10/doc/null]
java.lang.IllegalArgumentException: pipeline with id [filebeat-5.5.0-haproxy-haproxy_log-pipeline] does not exist
at org.elasticsearch.ingest.PipelineExecutionService.getPipeline(PipelineExecutionService.java:194) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.ingest.PipelineExecutionService.access$100(PipelineExecutionService.java:41) ~[elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.ingest.PipelineExecutionService$2.doRun(PipelineExecutionService.java:88) [elasticsearch-5.5.0.jar:5.5.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.5.0.jar:5.5.0
]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.5.0.jar:5.5.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
I have run a curl command to verify this pipeline DOES exist on all elasticsearch nodes and it does.
Actually all of my pipelines are returning the same error. Very strange.
Also my elasticsearch cluster is configured as follows on all nodes:
discovery.zen.ping.unicast.hosts: es1,es2,es3,es4,es5,es6,es7,es8,es9,es10,es11
discovery.zen.minimum_master_nodes: 3
node.ingest: true