There are errors in the documentation to get started with creating your own beats. If you search in the history, they have been around for some time and haven't been updated. This makes it extremely difficult to create a new beat, especially for someone new to GOlang.
I've cloned the beats repo per the developer setup description. Since I'm also developing against 7.10.1, I've checked out that version to my local branch to run the following...
- Page: Generating Your Beat
The documentation says:
cd ${GOPATH}/src/github.com/{user}
Run the mage script to generate the custom beat:
mage GenerateCustomBeat
- This is incorrect. To run "mage GenerateCustomBeat" you need to be in the beats directory that was cloned. But you need to create your user directory or GenerateCustomBeat fails. The documentation should be something like this:
mkdir -p ${GOPATH}/src/github.com/{user}
cd ${GOPATH}/src/github.com/src/github.com/elastic/beats
mage GenerateCustomBeat
-
This isn't in the documentation, but I had issues with GenerateCustomBeat failing while trying to run setup.GitAdd from the file: ${GOPATH}/src/github.com/src/github.com/elastic/beats/generator/common/beatgen/beatgen.go
-
I fixed this by commenting out the following line:
mg.Deps(setup.GitAdd)
-
It would be nice to explain in the documentation why it's trying do create repository and add the new beat.
-
Also on this page the user input for the GenerateCustomBeat is not the same and some explanations should be added to the documentation.
Enter the beat name [examplebeat]: <--- Should the be first initial capitalized?
Enter your github name [your-github-name]: <--- This should be the user specified above.
Enter the beat path [github.com/{user}/examplebeat] <-- github.com/{user} directory needs to exist
Enter your full name [Firstname Lastname]: <--- no issues
Enter the beat type [beat]: <--- What is this beat type referring to here?
Enter the GitHub - elastic/beats: 🐠 Beats - Lightweight shippers for Elasticsearch & Logstash revision [master]: <--- Is this the version tag or the git hash for the version you want to use, like 1da173a for v7.10.1?
- If the beats revision is left as the default "master", the script fails:
# github.com/elastic/beats/v7/libbeat/common/transport/tlscommon
../../../../pkg/mod/github.com/elastic/beats/v7@v7.0.0-alpha2.0.20210203145817-10fdd24879c7/libbeat/common/transport/tlscommon/tls_config.go:112:3: unknown field 'VerifyConnection' in struct literal of type tls.Config
../../../../pkg/mod/github.com/elastic/beats/v7@v7.0.0-alpha2.0.20210203145817-10fdd24879c7/libbeat/common/transport/tlscommon/tls_config.go:123:4: unknown field 'VerifyConnection' in struct literal of type tls.Config
../../../../pkg/mod/github.com/elastic/beats/v7@v7.0.0-alpha2.0.20210203145817-10fdd24879c7/libbeat/common/transport/tlscommon/tls_config.go:141:4: unknown field 'VerifyConnection' in struct literal of type tls.Config
../../../../pkg/mod/github.com/elastic/beats/v7@v7.0.0-alpha2.0.20210203145817-10fdd24879c7/libbeat/common/transport/tlscommon/tls_config.go:149:8: config.VerifyConnection undefined (type *tls.Config has no field or method VerifyConnection)
note: module requires Go 1.15
Error: error compiling magefiles
make: *** [fields] Error 1
Error: running "make update" failed with exit code 2
- I've gotten it to run by using v7.10.1 as the beats revision...
- Page: Fetching Dependencies and Setting
The documentation says:
To fetch dependencies and set up the Beat, run:
cd ${GOPATH}/src/github.com/{user}/countbeat
make setup
- This is incorrect, there is no target for setup, and was documented in this posting and should be:
make update
Now, when I run the 'mage build' command, I get the following output:
$ mage build
build: Building examplebeat
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'
Error: template: inline:1:128: executing "inline" at : error calling commit: running "git rev-parse HEAD" failed with exit code 128
failed to expand template '-s -X github.com/elastic/beats/v7/libbeat/version.buildTime={{ date }} -X github.com/elastic/beats/v7/libbeat/version.commit={{ commit }}'
github.com/elastic/beats/v7/dev-tools/mage.expandTemplate
/Users/jackemuk/go/pkg/mod/github.com/elastic/beats/v7@v7.10.1/dev-tools/mage/common.go:100
github.com/elastic/beats/v7/dev-tools/mage.Expand
/Users/jackemuk/go/pkg/mod/github.com/elastic/beats/v7@v7.10.1/dev-tools/mage/common.go:56
github.com/elastic/beats/v7/dev-tools/mage.MustExpand
/Users/jackemuk/go/pkg/mod/github.com/elastic/beats/v7@v7.10.1/dev-tools/mage/common.go:62
github.com/elastic/beats/v7/dev-tools/mage.Build
/Users/jackemuk/go/pkg/mod/github.com/elastic/beats/v7@v7.10.1/dev-tools/mage/build.go:139
github.com/elastic/beats/v7/dev-tools/mage/target/build.Build
/Users/jackemuk/go/pkg/mod/github.com/elastic/beats/v7@v7.10.1/dev-tools/mage/target/build/build.go:26
main.Build
/Users/jackemuk/go/src/github.com/jackemuk-es/examplebeat/magefile.go:82
main.main.func14
/Users/jackemuk/go/src/github.com/jackemuk-es/examplebeat/mage_output_file.go:490
main.main.func12.1
/Users/jackemuk/go/src/github.com/jackemuk-es/examplebeat/mage_output_file.go:275
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1373
Does anyone from Elastic read these boards? Someone posted a similar issue and it was automatically closed because nobody responded: issue creating new beat #260339