Special Character as URL Params

[sbruyere (Stephan Bruyere)]

Hello,

I try tu use the following URL in a Http_poller Input Plugin.
url => "http://10.0.10.10/login/login.cgi?mode=AUTH_LOGIN&params=userid|password&"

But the special character " | " (between userid and password as params value) makes logstash to stop.

Any idea how to declare this URL in a way that Logstash takes it ?

Br,

Stephan

[kopacko (Jason Kopacko)]

Grok Debugger is your friend. [ https://grokdebug.herokuapp.com/ ]

Using what you posted, I created this:

url => "%{URIPROTO:protocol}://%{IP:ip_address}%{URIPATH:url_path}%{URIPARAM:url_param}

Me personally, I like to split things out to as many fields as possible, especially the important ones like a username.

url => "%{URIPROTO:protocol}://%{IP:ip_address}%{URIPATH:url_path}.*mode=%{DATA:url_mode}&params=%{DATA:user_id}|%{GREEDYDATA:url_params}&

EDIT:

Forgot the output.

{
"protocol": [
[
"http"
]
],
"ip_address": [
[
"10.0.10.10"
]
],
"IPV6": [
[
null
]
],
"IPV4": [
[
"10.0.10.10"
]
],
"url_path": [
[
"/login/login.cgi"
]
],
"url_mode": [
[
"AUTH_LOGIN"
]
],
"user_id": [
[
"userid"
]
],
"url_params": [
[
"password"
]
]
}

[sbruyere (Stephan Bruyere)]

Hello Kopacko,

Thanks for your reply but my problem is about the URL setting of the Http_Poller Input Plugin, not the Grog Filter Plugin.

When I give to the URL Setting a different URL it works well. As soon as I have this character " |", my logstash instance stops.

Yo,

[kopacko (Jason Kopacko)]

Wow, haha, my bad.

I was thinking you are parsing an incoming log. Sorry about that.

[magnusbaeck (Magnus Bäck)]

Are there any clues in Logstash's log? Have you tried URL encoding the pipe (i.e. %7C)?

[sbruyere (Stephan Bruyere)]

With the vertical Bar in the Url, logstash stops with Error message : "Invalid URL".
I've tried "...userid%7Cpassword" then the URL is issued but "%7C" is included as-is in the Request. Seen with a sniffer... %7C is not decoded.
I've also URL encoded the entire URL then it became a invalid URL again.
Other idea ?

[sbruyere (Stephan Bruyere)]

No OK I got it. The %7C is meant to be decoded by the server and indeed it does the job.
Thank you very much. Sorry for the Hysteresis in my understanding