Hello,
I am trying to use kubernetes autodiscover with multiline pattern matching. It seems working for some java stacktrace but not for all. I found some logs having issue with multiline matching have backtick in the content.
Do I have a way to debug multiline matching in filebeat 6.6.0 ?
For example theses log lines are not grouped together :
2019-02-13 10:55:40.441 432f2c313174a7e737a975bfbeab2b21 ERROR 1 --- [nio-8080-exec-5] c.c.c.ws.dao.DatawarehouseDao : Les données du rapport tauxPerteStructure n'ont pas pu être récupérés.
org.springframework.web.client.RestClientException: Error while extracting response for type [class com.mycompany.myproject.ws.domain.dw.DWResponse] and content type [application/json;charset=UTF-8]; nested exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot construct instance of `com.mycompany.myproject.ws.domain.dw.DWResponse` (although at least one Creator exists): no int/Int-argument constructor/factory method to deserialize from Number value (406); nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `com.mycompany.myproject.ws.domain.dw.DWResponse` (although at least one Creator exists): no int/Int-argument constructor/factory method to deserialize from Number value (406)
at [Source: (PushbackInputStream); line: 1, column: 1]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:115) ~[spring-web-5.0.10.RELEASE.jar!/:5.0.10.RELEASE]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:959) ~[spring-web-5.0.10.RELEASE.jar!/:5.0.10.RELEASE]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:942) ~[spring-web-5.0.10.RELEASE.jar!/:5.0.10.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:689) ~[spring-web-5.0.10.RELEASE.jar!/:5.0.10.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:644) ~[spring-web-5.0.10.RELEASE.jar!/:5.0.10.RELEASE]
This is the configuration I use with filbeat 6.6.0 :
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
labels:
app: filebeat
data:
filebeat.yml: |-
filebeat:
config:
modules:
path: ${path.config}/modules.d/*.yml
autodiscover:
providers:
- type: kubernetes
templates:
- condition:
equals:
kubernetes.labels.type: java
config:
- type: docker
containers.ids:
- "${data.kubernetes.container.id}"
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2} '
multiline.negate: true
multiline.match: after
processors:
- add_kubernetes_metadata:
in_cluster: true
- condition:
contains:
kubernetes.labels.app: nginx
config:
- module: nginx
access:
input:
type: docker
containers.stream: stdout
containers.ids:
- "${data.kubernetes.container.id}"
processors:
- add_kubernetes_metadata:
in_cluster: true
error:
input:
type: docker
containers.stream: stderr
containers.ids:
- "${data.kubernetes.container.id}"
processors:
- add_kubernetes_metadata:
in_cluster: true
- config:
- type: docker
containers.ids:
- "${data.kubernetes.container.id}"
processors:
- add_kubernetes_metadata:
in_cluster: true
setup:
template:
name: "filebeat"
pattern: "filebeat-*"
output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
index: "%{[kubernetes.namespace]:nonamespace}-filebeat-%{+yyyy.MM.dd}"
pipelines:
- pipeline: java-logs-pipeline
when.equals:
kubernetes.labels.type: java
- pipeline: mongodb-logs-pipeline
when.equals:
kubernetes.labels.app: mongo-pod
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-prospectors
labels:
app: filebeat
data:
kubernetes.yml: |-
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-modules
labels:
app: filebeat
data:
nginx.yml: |-
- module: nginx
access:
enabled: true
error:
enabled: true