Logs are not coming from the client when custom logs integration was added

Elastic Stack Elastic Agent
1

I was trying to use custom logs to ingest log files of tomcat9 from a client system.

*The client system is using the policy Agent policy 1
*The client system is also running apache2. Apache httpd server integration (integration name: apache-2) was used to collect the data from apache.
I already used a custom logs integration (integration name: tomcat-access-test) to ingest logs from /var/log/tomcat9/localhost_access-logs from the client system in the above policy (Agent policy 1) and it was working.

Now when I tried to add another custom logs integration (integration name: tomcat-catalina-log) in the Agent policy 1 to get logs from /var/log/tomcat9/catalina* of the client system, no logs were coming from the client. Even the logs from apache-2 integration stopped.
When I remove the newly created custom integration (tomcat-catalina-log), I could get the logs from other integrations (apache-2 and tomcat-access-test).

The custom logs integration I used for catalina logs is as follows:

Integration name: tomcat-catalina-logs
Namespace: default
Log file path: /var/log/tomcat9/catalina*
Dataset name: bbb-tomcat-catalina-logs
Custom configurations:
        pipeline: 'bbb-tomcat-catalina-test-pipeline'
        multiline.type: pattern
        multiline.pattern: '^[[:space:]]'
        multiline.negate: false
        multiline.match: after

The pipeline used (bbb-tomcat-catalina-test-pipeline) is given below:

[
  {
    "dissect": {
      "field": "message",
      "pattern": "%{date} %{time} %{catalina.warnLevel} [%{catalina.method}] %{catalina.class} %{catalina.logMessage}",
      "ignore_missing": true
    }
  }
]

The index template (bbb-tomcat-catalina-log-test) used is given below:

{
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "aaa-tomcat-test-policy"
        },
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_hot"
            }
          }
        }
      }
    },
    "mappings": {
      "properties": {
        "catalina": {
          "properties": {
            "class": {
              "type": "keyword"
            },
            "logMessage": {
              "type": "text",
              "index_options": "docs",
              "norms": false
            },
            "method": {
              "type": "keyword"
            },
            "warnLevel": {
              "type": "keyword"
            }
          }
        },
        "date": {
          "type": "keyword"
        },
        "time": {
          "type": "keyword"
        }
      }
    },
    "aliases": {}
  }
}

The index pattern used was logs-bbb-tomcat-catalina-logs*

Can someone help ?

Thanks

2

The logs are ingested when I restart the elastic-agent on the client system. The logs after restart is not coming.
Every restart of elastic-agent ingests the logs on the client system.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.