# Custom message filter with logstash
**URL:** https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626
**Category:** Logstash
**Created:** [May 11, 2021, 3:14am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626 "2021-05-11T03:14:51Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@cuongnp](https://discuss.elastic.co/u/cuongnp)
#### Post date: [May 11, 2021, 3:14am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/1 "2021-05-11T03:14:51Z")
Hi all,
I have a message log below:
\<85\>1 2021-05-11T09:25:02+07:00 172.19.16.241 CP-GW - Log [Fields@1.3.6.1.4.1.2620 Log delay="1620699902" src="172.19.9.18" dst="172.19.11.13" proto="6" UP\_match\_table="TABLE\_START" ROW\_START="0" match\_id="35" layer\_uuid="f5cec687-05e5-4573-b1dc-08119f24cbc9"
="Network" rule\_uid="21e1b03b-5e20-4913-9d84-468164e67d8a" rule\_name="" ROW\_END="0" UP\_match\_table="TABLE\_END" ProductName="VPN-1 & FireWall-1" svc="10051" sport\_svc="64308" ProductFamily="Network" ].
I would like to define field for this message with output like this:
{
timestamp =\> 2021-05-11T09:25:02+07:00
src\_ip =\> 172.19.9.18
dst\_ip =\> 172.19.11.13
UP\_match\_table =\> TABLE\_START
ROW\_START =\> 0
match\_id =\> 35
layer\_uuid =\> f5cec687-05e5-4573-b1dc-08119f24cbc9
layer\_name =\> Network
rule\_uid =\> 21e1b03b-5e20-4913-9d84-468164e67d8a
rule\_name =\> " "
ROW\_END =\> 0
UP\_match\_table =\> TABLE\_END
ProductName =\> VPN-1 & FireWall-1
svc =\> 10051
sport\_svc =\> 64308
ProductFamily =\> Network
}
I read about logstash filter but I'm confused which filter I should use. Can you help me figure out
Thanks
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 11, 2021, 3:15am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/2 "2021-05-11T03:15:25Z")
You will want to use grok or dissect to do this for you.
---
### Author:  [@cuongnp](https://discuss.elastic.co/u/cuongnp)
#### Post date: [May 11, 2021, 4:41am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/3 "2021-05-11T04:41:02Z")
Thanks Mark,
I tried with dissect filter, this is my configuration for logstash config
input { stdin { } }
filter {
dissect {
mapping =\> { 'message' =\> '%{} %{timestamp} %{} %{appname} %{} %{} [%{} %{} %{} %{service\_id} %{ip\_source} %{ip\_dst} %{} %{up\_match\_table} %{row\_start} %{match\_id} %{layer\_uuid} %{layer\_name} %{rule\_uid} %{rule\_name} %{row\_end} %{end\_match\_table} %{product\_name} %{svc} %{sport\_svc} %{product\_family}]' }
}
}
output {
stdout { codec =\> rubydebug }
}
And I got output:
 ~
It created fields but value still contain field name, for example
"layer\_name" =\> "layer\_name="Network"" should be "layer\_name" =\> "Network" instead
Thanks
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 11, 2021, 4:47am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/4 "2021-05-11T04:47:58Z")
Please don't post pictures of text, they are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them 🙂
---
### Author:  [@cuongnp](https://discuss.elastic.co/u/cuongnp)
#### Post date: [May 11, 2021, 5:53am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/5 "2021-05-11T05:53:08Z")
Hi Mark, I repost the output, please have a look
\<85\>1 2021-05-11T11:13:43+07:00 172.19.16.243 CP-GW - Log [Fields@1.3.6.1.4.1.2620 Log delay="1620706423" service\_id="http" src="172.19.15.14" dst="172.19.16.22" proto="6" UP\_match\_table="TABLE\_START" ROW\_START="0" match\_id="39" layer\_uuid="f5cec687-05e5-4573-b1dc-08119f24cbc9" layer\_name="Network" rule\_uid="eb1a5e3a-4f76-4a60-a3e1-9221b505a422" rule\_name="TORtoFW" ROW\_END="0" UP\_match\_table="TABLE\_END" ProductName="VPN-1 & FireWall-1" svc="80" sport\_svc="59304" ProductFamily="Network"]
{
"layer\_name" =\> "layer\_name="Network"",
"sport\_svc" =\> "FireWall-1"",
"appname" =\> "CP-GW",
"row\_end" =\> "ROW\_END="0"",
"ip\_dst" =\> "dst="172.19.16.22"",
"@version" =\> "1",
"@timestamp" =\> 2021-05-11T04:35:47.407Z,
"product\_family" =\> "svc="80" sport\_svc="59304" ProductFamily="Network" ",
"layer\_uuid" =\> "layer\_uuid="f5cec687-05e5-4573-b1dc-08119f24cbc9"",
"product\_name" =\> "ProductName="VPN-1",
"timestamp" =\> "2021-05-11T11:13:43+07:00",
"ip\_source" =\> "src="172.19.15.14"",
"rule\_uid" =\> "rule\_uid="eb1a5e3a-4f76-4a60-a3e1-9221b505a422"",
"rule\_name" =\> "rule\_name="TORtoFW"",
"host" =\> "xplat-mon-01",
"row\_start" =\> "ROW\_START="0"",
"svc" =\> "&",
"up\_match\_table" =\> "UP\_match\_table="TABLE\_START"",
"end\_match\_table" =\> "UP\_match\_table="TABLE\_END"",
"match\_id" =\> "match\_id="39"",
"message" =\> "\<85\>1 2021-05-11T11:13:43+07:00 172.19.16.243 CP-GW - Log [Fields@1.3.6.1.4.1.2620 Log delay="1620706423" service\_id="http" src="172.19.15.14" dst="172.19.16.22" proto="6" UP\_match\_table="TABLE\_START" ROW\_START="0" match\_id="39" layer\_uuid="f5cec687-05e5-4573-b1dc-08119f24cbc9" layer\_name="Network" rule\_uid="eb1a5e3a-4f76-4a60-a3e1-9221b505a422" rule\_name="TORtoFW" ROW\_END="0" UP\_match\_table="TABLE\_END" ProductName="VPN-1 & FireWall-1" svc="80" sport\_svc="59304" ProductFamily="Network"]",
"service\_id" =\> "service\_id="http""
Thanks
---
### Author:  [@cuongnp](https://discuss.elastic.co/u/cuongnp)
#### Post date: [May 11, 2021, 7:02am UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/6 "2021-05-11T07:02:24Z")
It came out with "layer\_name" =\> "layer\_name="Network"", and I would like to ignore the specific text before "Network", so the result should be "layer\_name" =\> "Network"
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 11, 2021, 5:31pm UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/7 "2021-05-11T17:31:47Z")
You might find it easier to use
```
dissect { mapping => { 'message' => '%{} %{timestamp} %{} %{appname} %{} %{} [%{[@metadata][restOfLine]}]' } remove_field => ["message"] }
kv { source => "[@metadata][restOfLine]" }
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 8, 2021, 5:32pm UTC](https://discuss.elastic.co/t/custom-message-filter-with-logstash/272626/8 "2021-06-08T17:32:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.