Hi,
In de default integration 'system' you can add 'system syslog logs' to collect. I added a few logs here. That's working.
But every 2 days or so this logging stops. A workaround is to edit the interation, change nothing and save again. This forces a redeploy on every agent and logs are coming in again.
There is an error in the elastic-agent logging round the time logs stopped:
'{"log.level":"info","@timestamp":"2021-03-12T13:33:38.672+0100","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":48},"message":"New State ID is Negq_6m6","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:38.674+0100","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":49},"message":"Converging state requires execution of 4 step(s)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:38.675+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/metricbeat-7.11.1-linux-x86_64.tar.gz. Skipping operation ope>
{"log.level":"info","@timestamp":"2021-03-12T13:33:38.675+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/metricbeat-7.11.1-linux-x86_64.tar.gz. Skipping operation ope>
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.204+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-install' skipped for metricbeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.205+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-start' skipped for metricbeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.215+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"endpoint-security.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/endpoint-security-7.11.1-linux-x86_64.tar.gz. Skipping>
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.216+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"endpoint-security.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/endpoint-security-7.11.1-linux-x86_64.tar.gz. Skipping>
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.897+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-install' skipped for endpoint-security.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.899+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-start' skipped for endpoint-security.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.901+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/filebeat-7.11.1-linux-x86_64.tar.gz. Skipping operation operati>
{"log.level":"info","@timestamp":"2021-03-12T13:33:39.901+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/filebeat-7.11.1-linux-x86_64.tar.gz. Skipping operation operati>
{"log.level":"info","@timestamp":"2021-03-12T13:33:40.280+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:40+01:00: type: 'STATE': sub_type: 'RUNNING' message: Application: endpoint-security--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to RUN>
{"log.level":"info","@timestamp":"2021-03-12T13:33:40.866+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-install' skipped for filebeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:40.867+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-start' skipped for filebeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:40.868+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/filebeat-7.11.1-linux-x86_64.tar.gz. Skipping operation operati>
{"log.level":"info","@timestamp":"2021-03-12T13:33:40.868+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/filebeat-7.11.1-linux-x86_64.tar.gz. Skipping operation operati>
{"log.level":"warn","@timestamp":"2021-03-12T13:33:41.320+0100","log.origin":{"file.name":"status/reporter.go","file.line":146},"message":"Elastic Agent status changed to: 'degraded'","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2021-03-12T13:33:41.355+0100","log.origin":{"file.name":"log/reporter.go","file.line":36},"message":"2021-03-12T13:33:41+01:00: type: 'ERROR': sub_type: 'FAILED' message: Application: filebeat--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to FAILED: 1 er>
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.355+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:41+01:00: type: 'STATE': sub_type: 'STARTING' message: Application: filebeat--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to STARTING: S>
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.356+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:41+01:00: type: 'STATE': sub_type: 'STARTING' message: Application: filebeat--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to RESTARTING:>
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.567+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-install' skipped for filebeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.568+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-start' skipped for filebeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.569+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/metricbeat-7.11.1-linux-x86_64.tar.gz. Skipping operation ope>
{"log.level":"info","@timestamp":"2021-03-12T13:33:41.570+0100","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.7.11.1 already exists in /var/lib/elastic-agent/data/elastic-agent-9b2fec/downloads/metricbeat-7.11.1-linux-x86_64.tar.gz. Skipping operation ope>
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.466+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:42+01:00: type: 'STATE': sub_type: 'STARTING' message: Application: filebeat--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to STARTING: S>
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.947+0100","log.origin":{"file.name":"status/reporter.go","file.line":146},"message":"Elastic Agent status changed to: 'online'","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.947+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:42+01:00: type: 'STATE': sub_type: 'RUNNING' message: Application: filebeat--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to RUNNING: Run>
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.959+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-install' skipped for metricbeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.960+0100","log.origin":{"file.name":"operation/operator.go","file.line":245},"message":"operation 'operation-start' skipped for metricbeat.7.11.1","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:42.961+0100","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":66},"message":"Updating internal state","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2021-03-12T13:33:49.860+0100","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2021-03-12T13:33:49+01:00: type: 'STATE': sub_type: 'RUNNING' message: Application: endpoint-security--7.11.1[b2daeca0-7d11-11eb-a2ac-47c38a57d10e]: State changed to RUN'