Hello,
Below is my java GC log file sample:
[2024-10-14T00:39:07.265+0000][6.567s][gc,start ] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
[2024-10-14T00:39:07.267+0000][6.568s][gc,task ] GC(0) Using 4 workers of 4 for evacuation
[2024-10-14T00:39:07.277+0000][6.579s][gc,phases ] GC(0) Pre Evacuate Collection Set: 0.2ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,phases ] GC(0) Post Evacuate Cleanup 2: 0.3ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,phases ] GC(0) Other: 1.6ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,heap ] GC(0) Humongous regions: 0->0
[2024-10-14T00:39:07.279+0000][6.581s][gc,metaspace] GC(0) Metaspace: 12174K(12480K)->12174K(12480K) NonClass: 10915K(11072K)->10915K(11072K) Class: 1259K(1408K)->1259K(1408K)
[2024-10-14T00:39:07.279+0000][6.581s][gc ] GC(0) Pause Young (Normal) (G1 Evacuation Pause) 51M->10M(1032M) 13.996ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,cpu ] GC(0) User=0.03s Sys=0.01s Real=0.02s
[2024-10-14T00:39:07.265+0000][6.567s][gc,start ] GC(1) Pause Young (Normal) (G1 Evacuation Pause)
[2024-10-14T00:39:07.267+0000][6.568s][gc,task ] GC(1) Using 4 workers of 4 for evacuation
[2024-10-14T00:39:07.277+0000][6.579s][gc,phases ] GC(1) Pre Evacuate Collection Set: 0.2ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,phases ] GC(1) Post Evacuate Cleanup 2: 0.3ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,phases ] GC(1) Other: 1.6ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,heap ] GC(1) Humongous regions: 0->0
[2024-10-14T00:39:07.279+0000][6.581s][gc,metaspace] GC(1) Metaspace: 12174K(12480K)->12174K(12480K) NonClass: 10915K(11072K)->10915K(11072K) Class: 1259K(1408K)->1259K(1408K)
[2024-10-14T00:39:07.279+0000][6.581s][gc ] GC(1) Pause Young (Normal) (G1 Evacuation Pause) 51M->10M(1032M) 13.996ms
[2024-10-14T00:39:07.279+0000][6.581s][gc,cpu ] GC(1) User=0.03s Sys=0.01s Real=0.02s
Above log reads like each GC(x) is one message which we need to send to logstash. My start text is always gc,start. So configured below in filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- "/opt/micro_service/TA/MOUNT/gc_logs/s1gc.log"
multiline.pattern: 'gc,start(.)*?'
multiline.negate: true
multiline.match: after
Now, if you look at the log, lines containing gc,phases will be a lot which i removed in the sample provided above. What is happening is sometimes, due to more number of lines, filebeats is not taking the full event as a message, missing last few lines. Pls let me know