When I package my beat based on metricbeat like this and I install this beat on debian 7 (wheezy) the beat starts and works like expected but it refuses to run in the background when started via inet.d. If I install metricbeat on the same system metricbeat run perfectly in background when started via inet.d. I compared the inet.d startup scripts of both beats and these are exactly the same.
Does anybody has an explanation for this? Has this something to do the way I package my beat (without CGO support)? All thoughts are welcome.
As you can see they are quite similar except from the fact that in the log entries of my beat there are showing up DEBUG messages. This makes me suspecting that there is some kind of flag to be set at compile time so that the beat will run as a daemon instead of running in the foreground. I searched the code but could not find any such flag or something similar. Any comments?
Did you install both beats the same way, meaning with the deb package?
There is a difference between the "raw" binaries and the one installed through the packages. They are wrapped into a god (go-deamon) to be running in the background.
Do you get some kind of error? The daemonizing is handled by go-daemon when using SysV (inet.d). You'll see that the init script calls a {{beatname}}-god binary.
I found the cause of the problem or at least a part of it. Running /usr/share/smartbeat/bin/smartbeat -h reveals this output:
Usage of /usr/share/smartbeat/bin/smartbeat:
-E value
Configuration overwrite
-N Disable actual publishing for testing
-c value
Configuration file, relative to path.config (default beat.yml)
-configtest
Test configuration and exit.
-cpuprofile string
Write cpu profile to file
-d value
Enable certain debug selectors
-e Log to stderr and disable syslog/file output
-httpprof string
Start pprof http server
-httptest.serve string
if non-empty, httptest.NewServer serves on this address and blocks
-memprofile string
Write memory profile to this file
-path.config value
Configuration path
-path.data value
Data path
-path.home value
Home path
-path.logs value
Logs path
-setup
Load the sample Kibana dashboards
-strict.perms
Strict permission checking on config files (default true)
-system.hostfs string
mountpoint of the host's filesystem for use in monitoring a host from within a container
-v Log at INFO level
-version
Print the version and exit
there are no test and config options available like in $TEST_ARGS in the init.d script of smartbeat. Replacing test and config in the init.d script by -configtest resolves the issue.
But.... Starting my own beat this way outputs:
/etc/init.d/smartbeat start
2018-03-15T10:20:13.610+0100 INFO instance/beat.go:468 Home path: [/usr/share/smartbeat/bin] Config path: [/usr/share/smartbeat/bin] Data path: [/usr/share/smartbeat/bin/data] Logs path: [/usr/share/smartbeat/bin/logs]
2018-03-15T10:20:13.611+0100 INFO instance/beat.go:475 Beat UUID: d647fa4d-b908-4d0e-ba31-703cb82a2cd9
2018-03-15T10:20:13.615+0100 INFO instance/beat.go:213 Setup Beat: smartbeat; Version: 6.2.3
2018-03-15T10:20:13.615+0100 WARN instance/metrics_other.go:8 Metrics not implemented for this OS.
2018-03-15T10:20:13.617+0100 INFO pipeline/module.go:76 Beat name: puppetwheezy
2018-03-15T10:20:13.618+0100 WARN status/status.go:49 EXPERIMENTAL: The gearmand status metricset is experimental
2018-03-15T10:20:13.618+0100 WARN [cfgwarn] instance/beat.go:283 DEPRECATED: -configtest flag has been deprecated, use configtest subcommand Will be removed in version: 6.0
Config OK
2018-03-15T10:20:13.618+0100 INFO [monitoring] log/log.go:97 Starting metrics logging every 30s
2018-03-15T10:20:13.618+0100 INFO [monitoring] log/log.go:132 Total non-zero metrics {"monitoring": {"metrics": {"libbeat":{"config":{"module":{"running":0}},"output":{"type":"redis"},"pipeline":{"clients":0,"events":{"active":0}}}}}}
2018-03-15T10:20:13.619+0100 INFO [monitoring] log/log.go:133 Uptime: 20.734141ms
2018-03-15T10:20:13.619+0100 INFO [monitoring] log/log.go:110 Stopping metrics logging.
There is a contradiction in it as you can see the version of my beat is 6.2.3 while there's a message saying:
2018-03-15T10:20:13.618+0100 WARN [cfgwarn] instance/beat.go:283 DEPRECATED: -configtest flag has been deprecated, use configtest subcommand Will be removed in version: 6.0
As far as I know I started building my beat against version 6.0 of the metricbeat library, in the mean time I upgraded towards version 6.2 of the metricbeat library but I looks like there were libraries used from before version 6.0 of the metricbeat library. Any clues of what could be (went) wrong???
The first thing that popped up in my head is that there could be potentially a different version of libbeat/metricbeat under GOPATH/src/github.com/elastic/beats and GOPATH/src/yourbeat/vendor/github.com/elastic/beats. And for some parts it uses the code in vendor and for some parts the code outside?
I now took a whole new virtual machine, followed exactly this guide, generated the examplebeat and packaged this beat with make package. I took the file examplebeat-6.2.3-SNAPSHOT-amd64.deb and installed it on a Debian Wheezy OS, running /usr/share/examplebeat/bin/examplebeat -h gave this output:
/usr/share/examplebeat/bin/examplebeat -h
Usage of /usr/share/examplebeat/bin/examplebeat:
-E value
Configuration overwrite
-N Disable actual publishing for testing
-c value
Configuration file, relative to path.config (default beat.yml)
-configtest
Test configuration and exit.
-cpuprofile string
Write cpu profile to file
-d value
Enable certain debug selectors
-e Log to stderr and disable syslog/file output
-httpprof string
Start pprof http server
-httptest.serve string
if non-empty, httptest.NewServer serves on this address and blocks
-memprofile string
Write memory profile to this file
-path.config value
Configuration path
-path.data value
Data path
-path.home value
Home path
-path.logs value
Logs path
-plugin value
Load additional plugins
-setup
Load the sample Kibana dashboards
-strict.perms
Strict permission checking on config files (default true)
-system.hostfs string
mountpoint of the host's filesystem for use in monitoring a host from within a container
-v Log at INFO level
-version
Print the version and exit
Running the command /etc/init.d/examplebeat start also did not make the beat running in the background. Can you have a look if you can reproduce this?
I think you can stop searching. The explanation can be found in this topic.
To summarize the init.d script of my beat (as compiled with the make scripts provided in the beat library) uses the subcommand config test to test the config at startup, but as explained in the referenced topic the subcommands aren't available in custom generated beats because the generator doesn't take them into account.
As a workaround I modified the /etc/init.d startup script to use -configtest instead of test config. But anyway this is some kind of a bug. Or the subcommands need to be available in custom generated beats or the inet.d startup scripts need to use the 'old' parameters instead of the subcommands to test the configfile at startup.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.