Logstash parse out \r and \t entries

Hello there,

I am having some issues with parsing out some logstash data. In my data, I have \r's and \t's that are floating around that I need to get rid of.

\tLogon ID:\t\t(0x2,0x50722C31)\r

I started trying to remove the \t's first so I made a kv

kv { 
source => [message]
trim => "\t"

but I found out that it's some sort of regular expression and I had to use another \ to escape the first one. I tried instead using \\t but then it removed all lower case t's and the 's instead of just the \t's :sad: I also tried \\\t but that only took out the \ and left the t's. Lastly, I tried \\\\t but it took me back to the \\t results.

If anyone could offer help, that would be appreciated.


In your data do you have tab characters or literal backslash lowercase tee?

There are literal tabs in the data. I think it's being converted into json and is shipped as pain text, which is making this complicated. I have NxLog shipping the logs to LogStash.

Assuming you are on UNIX, you can put a literal tab (ctrl/v ctrl/I) or carriage return (ctrl/v ctrl/m) in the logstash configuration file.

I can try. They data doesn't show up in elasticsearch with tabs though, even if I have a blank filter just to see how the data shows up. It shows up as

\tUser Name:\t\taccount\r so there isn't literal spaces

If I click on the JSON part of the data, there is an extra \ at each one.

\\tUser Name:\\t\\taccount\\r

I'll give your suggestion a go though.

Just tried your suggestion and it didn't seem to do any parsing. Assuming you mean pressing ctrl and v then ctrl and l it showed up as ^V^L in the config file. I am using CentOS btw

Tab is ctrl/eye, not ctrl/ell. I am assuming that lnext is set to ctrl/v. Does 'stty -a' show you that?

stty -a shows lnext = ^V

I substituted the i's instead of the L's and reran the services, but the \t's and \r's are still showing

Here is the entry, in JSON, inside elastic. Not sure if this helps or not

"User Logoff:\\r,\\r,\\tUser Name:\\tadmin\\r,\\r,\\tDomain:\\t\\tsrv-tst\\r,\\r,\\tLogon ID:\\t\\t(0x2,0x50722C31)\\r,\\r,\\tLogon Type:\\t5\\r,"

That really suggests to me that you have backslash + r or t in your message.

mutate { gsub => [ "message", "\\[rt]", "" ] }

would cure that.

Yeah I guess that what I was trying to say. In the original CSV, there are new lines, but NxLog seemed to replace those lines with plain-text JSON formatting for those, so the tabs were \t and the returns were \r, but as plain-text instead of code.

That worked like a charm @Badger. You are the boss!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.