I mucked around a bit more over the weekend, trying to figure out how to get dashboards added too. (The docs skipped a bit, I finally found some examples in the official metricbeat repo)
So, first I'm going to delete everything through the console:
DELETE _template/speedbeat-8.0.0
DELETE speedbeat-8.0.0*
And double checked the Index Management and Index Patterns management UI to make sure I didn't miss anything.
Then a fresh make & build
make update
// a quick jump to my text editor to set the kibana and elastic hosts since I don't dev against localhost
mage build && ./speedbeat -e -d "*"
From here the log lines between startup and first event that seem relevant are where it properly connects and creates a new index:
2020-09-14T17:27:58.156-0700 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'speedbeat-8.0.0' as ILM is enabled.
2020-09-14T17:27:58.156-0700 INFO eslegclient/connection.go:99 elasticsearch url: http://elasticservername:9200
Then it sends an event (and the payload debug output right above it shows the correct blob)
2020-09-14T17:28:46.091-0700 INFO beater/Speedbeat.go:85 Event sent
2020-09-14T17:28:47.096-0700 INFO [publisher_pipeline_output] pipeline/output.go:143 Connecting to backoff(elasticsearch(http://elasticservername:9200))
Then the ILM policy looks like it is attempting to create it fine, but I'd forgotten to manually delete the old policy
|2020-09-14T17:28:47.265-0700|INFO|[index-management]|idxmgmt/std.go:261|Auto ILM enable success.|
|2020-09-14T17:28:47.268-0700|INFO|[index-management.ilm]|ilm/std.go:139|do not generate ilm policy: exists=true, overwrite=false|
|2020-09-14T17:28:47.268-0700|INFO|[index-management]|idxmgmt/std.go:274|ILM policy successfully loaded.|
|2020-09-14T17:28:47.268-0700|INFO|[index-management]|idxmgmt/std.go:407|Set setup.template.name to '{speedbeat-8.0.0 {now/d}-000001}' as ILM is enabled.|
|2020-09-14T17:28:47.268-0700|INFO|[index-management]|idxmgmt/std.go:412|Set setup.template.pattern to 'speedbeat-8.0.0-*' as ILM is enabled.|
|2020-09-14T17:28:47.268-0700|INFO|[index-management]|idxmgmt/std.go:446|Set settings.index.lifecycle.rollover_alias in template to {speedbeat-8.0.0 {now/d}-000001} as ILM is enabled.|
|2020-09-14T17:28:47.268-0700|INFO|[index-management]|idxmgmt/std.go:450|Set settings.index.lifecycle.name in template to {speedbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.|
Then it tries to load the template
2020-09-14T17:28:47.329-0700 INFO [template_loader] template/load.go:121 Try loading template speedbeat-8.0.0 to Elasticsearch
|2020-09-14T17:28:47.840-0700|INFO|[template_loader]|template/load.go:113|template with name 'speedbeat-8.0.0' loaded.|
|2020-09-14T17:28:47.840-0700|INFO|[index-management]|idxmgmt/std.go:298|Loaded index template.|
At this point I switch to the kibana dev console and do a GET _template/speedbeat*
and copy that output in to a text editor, find one of my fields, and see it isn't defined as expected.
I.e. the console output says (and appears to match the output of ./speedbeat export template > foo.txt
)
"properties" : {
"elapsed" : {
"type" : "long"
},
but my fields.yml has elapsed defined as
- name: download.elapsed
type: integer
format: duration
required: true
description: foo
The beat continues to run and moves on to the next event send. No additional elastic or kibana "loading stuff" type things occur. At this point no new entry is visible through the Index Patterns UI, and finding one of the documents in the * index shows all of the fields with the yellow exclamation point.