Add field based on another

Elastic Stack Logstash
1

Hello.
I'd like to know if there's any possibility to add new field based on request filed, but with some regex.

(%{NOTSPACE:request}[?#$]\S*)

i'd like to add this one as field "request_file" and the full output to stay in original filed "request"

There's nothing related in documentation (or maybe lacks).
Please give me some ideas.

2

grok perhaps? It is not clear what you really want to do.

3

Well, the original request for example

pattern:
%{NOTSPACE:request}

/api/v1/projects/details.php?&clickID=*********

should remain as request field

Need to add new one field called request_file

which should be taken from pattern with regex
pattern:

(%{NOTSPACE:request}[?#$]\S*)

And should display only the requested file, for example.

/api/v1/projects/details.php

4

I am not sure, and cannot even understand ELK stack. I've added this line to mutate filter.

add_field => { "request_file" => ["(%{NOTSPACE:request}[?#$]\S*)"] }

I don't even know if it's correct.

5

That is not going to work. I think you want grok. Assuming that the [request] field contains the string '/api/v1/projects/details.php?&clickID=*********' you could try

grok { match => { "request" => "%{NOTSPACE:request_file}\?" } }

or

grok { match => { "request" => "%{URIPATH:request_file}%{URIPARAM:request_params}" } }
6

Can i use more that one grok in filter ?

7

Yes.

8

It does the trick, thanks!
I have one more question, but maybe it's better to open new ticket?

9

Well. the task were changed again.

And i was forced to add $uri to nginx log format, and now i need some suggestion how to create a new pattern to display it.

New log format

server.example.com 123.123.123.123 [01/Oct/2020:09:52:08 +0000] "POST /ws/whitelabel.php HTTP/2.0" 200 27429 "https://example.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36" "command=getUserCampaigns&token=*********************************** " "-" "-" "-" 0.152 "/ws/whitelabel.php"

As you can see the request_file is now latest, how can I make it again to be added as field?

Is it like that?

%{NOTSPACE:request_file}

10

It works but not relevant anymore.
Anyway Thank You for tips.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.