Cannot index event publisher.Event

[alexus]

Hello World!

I'm using Elastic stack 6.6.1/6.7.1 and while following Enroll Beats in central management | Filebeat Reference [6.6] | Elastic, I've noticed few of beats have following WARN in /var/logfilebeat/filebeat:

2019-02-11T16:56:45.967Z WARN elasticsearch/client.go:523 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbf10883339619143, ext:45895108537813, loc:(*time.Location)(0x21be580)}, Meta:common.MapStr{"pipeline":"filebeat-6.6.0-system-auth-pipeline"}, Fields:common.MapStr{"event":common.MapStr{"dataset":"system.auth"}, "prospector":common.MapStr{"type":"log"}, "input":common.MapStr{"type":"log"}, "fileset":common.MapStr{"module":"system", "name":"auth"}, "host":common.MapStr{"name":"kibana17"}, "offset":398726, "log":common.MapStr{"file":common.MapStr{"path":"/var/log/auth.log"}}, "message":"Feb 11 16:56:44 kibana17 sshd[31472]: Received disconnect from 106.13.0.129 port 49328:11: Bye Bye [preauth]", "beat":common.MapStr{"hostname":"kibana17", "version":"6.6.0", "name":"kibana17"}, "source":"/var/log/auth.log"}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc421bc7790), Source:"/var/log/auth.log", Offset:398835, Timestamp:time.Time{wall:0xbf1087a936e4817b, ext:45343066787428, loc:(*time.Location)(0x21be580)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x61550, Device:0x801}}}, Flags:0x1} (status=400): {"type":"illegal_argument_exception","reason":"pipeline with id [filebeat-6.6.0-system-auth-pipeline] does not exist"}

Please advise.

[rugenl (Len Rugen)]

Same issue....

[subhas (Subhas Takhellambam)]

I am having the same issue. Any update plz?

[steffens (Steffen Siering)]

Have you run filebeat setup after filebeat enroll?

[alexus]

I ran following command before encountering above error and creating new topic to discuss:

# filebeat setup --pipelines
# echo $?
0
#

[pierhugues (Pier-Hugues Pellerin)]

Looking at the issue I see that the pipeline is not correctly installed, we are working to automate the installation of pipelines, dashboard and template via the UI. But for now you will have to run.

./filebeat setup  --pipelines --modules system   

[alexus]

# filebeat setup --pipelines --modules system
Exiting: (assert) config is nil
Trace:goroutine 1 [running]:
runtime/debug.Stack(0x1785394, 0xb, 0xc4205c3360)
	/usr/local/go/src/runtime/debug/stack.go:24 +0xa7
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.raiseCritical(0x1860060, 0xc4200b8c60, 0xc42058b520, 0x16, 0xc42017aba0, 0xc420211620)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/error.go:151 +0x66
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.raiseNil(0x1860060, 0xc4200b8c60, 0x4, 0xc420245200)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/error.go:264 +0x3d
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.(*Config).Unpack(0x0, 0x156f620, 0xc42000c1c0, 0xc4202783c0, 0x4, 0x4, 0x101, 0xc42000c1c0)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/reify.go:130 +0x1fb
github.com/elastic/beats/libbeat/common.(*Config).Unpack(0x0, 0x156f620, 0xc42000c1c0, 0x14e9a11, 0x160bc20)
	/go/src/github.com/elastic/beats/libbeat/common/config.go:180 +0x63
github.com/elastic/beats/libbeat/outputs.ReadHostList(0x0, 0x160bc20, 0x188dcc0, 0x167a7c0, 0x188dc00, 0x14e9a03)
	/go/src/github.com/elastic/beats/libbeat/outputs/hosts.go:33 +0x88
github.com/elastic/beats/libbeat/outputs/elasticsearch.NewElasticsearchClients(0x0, 0xc42023a790, 0xc4204633a8, 0x0, 0x0, 0x0)
	/go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/elasticsearch.go:234 +0x43
github.com/elastic/beats/libbeat/outputs/elasticsearch.NewConnectedClient(0x0, 0x5, 0xc42048e320, 0x5)
	/go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/elasticsearch.go:208 +0x43
github.com/elastic/beats/filebeat/beater.(*Filebeat).setupPipelineLoaderCallback.func1(0x0, 0x17df448, 0x1866100)
	/go/src/github.com/elastic/beats/filebeat/beater/filebeat.go:166 +0x40
github.com/elastic/beats/libbeat/cmd/instance.(*Beat).Setup.func1(0xc4200deb40, 0x17df448, 0x1000000, 0x0, 0x0)
	/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:521 +0x37c
github.com/elastic/beats/libbeat/cmd/instance.(*Beat).Setup(0xc4200deb40, 0x17df448, 0x1000000, 0x0, 0x0)
	/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:537 +0x66
github.com/elastic/beats/libbeat/cmd.genSetupCmd.func1(0xc42022c6c0, 0xc42023a580, 0x0, 0xb)
	/go/src/github.com/elastic/beats/libbeat/cmd/setup.go:64 +0x269
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).execute(0xc42022c6c0, 0xc420274200, 0xb, 0x10, 0xc42022c6c0, 0xc420274200)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:704 +0x2c6
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4200f8280, 0x0, 0xc4200f8280, 0x178276f)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:785 +0x2e4
github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4200f8280, 0xc4204e7f78, 0xc4200ae058)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:738 +0x2b
main.main()
	/go/src/github.com/elastic/beats/x-pack/filebeat/main.go:22 +0x2f


#

also, per my early comment, there was no error running without --module system part.