Remove ^M from message

Elastic Stack Logstash
1

Hi,

I am trying to ingest Java exceptions into Elasticsearch via logstash.

In Kibana, they show up like this,

[2021-01-29 16:00:36.875] [139642182200000] [INF] : [58.122.202.198:57076] ==> RTSP OPTIONS Resp: RTSP/1.0 200 OK^M
CSeq: 1^M
Date: Fri Jan 29 16:00:36 2021^M
Server: ZooVod Streaming Server 1.3.0b^M
Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE^M
Content-Type: text/parameters^M
Content-Length: 50

^M
Video: 481,27
^M
Audio: 482,15


^M
^M

[2021-01-29 16:00:36.880] [139642182200000] [INF] : [58.122.202.198:57076] <== RTSP l[201] DESCRIBE rtsp://ecdnpoc-edge2.hanafostv.com:554/ontv/10300/T342592_1_201112_1339.ts.pac?id=747aT342592_1_201112_1339?passwd=4a107923834 RTSP/1.0~~User-Agent: Zooinnet SDK  BFX-UH-AT100_0.6~~CSeq: 2~~~~
[2021-01-29 16:00:36.880] [139642182200000] [INF] : [PathFileExist - 22] stat(/svc/data/btv/ontv/10300/T342592_1_201112_1339.ts.pac) not found, ret [-1]

[2021-01-29 16:00:36.885] [139642182200000] [INF] : [58.122.202.198:57076] --> Response SDP len [578]
[2021-01-29 16:00:36.936] [139642182200000] [INF] : [58.122.202.198:57076] <== RTSP l[247] SETUP rtsp://ecdnpoc-edge2.hanafostv.com:554/ontv/10300/T342592_1_201112_1339.ts.pac?id=747aT342592_1_201112_1339?passwd=4a107923834/ RTSP/1.0~~User-Agent: Zooinnet SDK  BFX-UH-AT100_0.6~~CSeq: 3~~Transport: RTP/AVP/TCP;unicast;interleaved=0-1~~~~
[2021-01-29 16:00:37.055] [139642182200000] [INF] : [58.122.202.198:57076] <== RTSP l[232] PLAY rtsp://ecdnpoc-edge2.hanafostv.com:554/ontv/10300/T342592_1_201112_1339.ts.pac?id=747aT342592_1_201112_1339?passwd=4a107923834 RTSP/1.0~~User-Agent: Zooinnet SDK  BFX-UH-AT100_0.6~~CSeq: 4~~Session: 83480~~Range: npt=0.000-~~~~

Is there a way to remove the ^M and make it more readable?

I did try,

mutate{
gsub => ["message", "^M", ""]
}

grok{
match => { "message" => "[%{TIMESTAMP_ISO8601:requestTime}] [%{NUMBER:longData:int}] [%{DATA:status}] : %{GREEDYDATA:msgData}"}
}

That didn't help. The message remained the same.

Appreciate your help on this.

Thanks,

2

Are you ingesting Windows logs on a machine that uses UNIX line endings? If so you will need a literal Ctrl/M in a mutate+gsub. On UNIX that would likely be Ctrl/V Ctrl/M (although config.support_escapes is another option that personally I would never use). If you are working on a Mac it would probably be something else.

3

yes, windows log

image
image1573×804 45.5 KB

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.