# Removing leading character and trailing 2 characters
**URL:** https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634
**Category:** Logstash
**Created:** [January 10, 2020, 4:29pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634 "2020-01-10T16:29:06Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 10, 2020, 4:29pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/1 "2020-01-10T16:29:06Z")
I am using kv to parse my fields due to the fields change depends on the logs that are sent from the siem.
```
kv {
value_split => “=’”
field_split => “’ “
```
which work fine but the very first field and the last. due to the extra characters. How do I remove them
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 10, 2020, 4:37pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/2 "2020-01-10T16:37:04Z")
It did not let me post the log trying again
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 10, 2020, 5:05pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/3 "2020-01-10T17:05:59Z")
You can use mutate+gsub to trim leading or trailing characters
```
"someField", "^.", ""
```
will remove a leading character
```
"someField", "..$", ""
```
will remove two trailing characters.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 10, 2020, 5:59pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/4 "2020-01-10T17:59:59Z")
I am totally a newbie on this. I am not so worried about the entryid field but the device field is required here is my logstash.conf file I commented the gsub lines out after they did not change anything for me.
I am not sure those fields are defined yet and for the devices field what comes out is
Tag IPgoesHERE and the value is /\>
```
input {
tcp {
port => 5142
type => "ossim-events"
codec => json {
charset => "CP1252"
}
}
syslog {
type => "syslog"
}
}
filter {
mutate {
# gsub => [
# "someField", "^.",""
# "someField", "..$",""
# ]
add_field => { "Agent_IP" => "%{host}"}
}
################### ALIENVAULT OSSIM Logs ###########################
if [type] == "ossim-events" {
kv {
value_split => "='"
field_split => "' "
}
}
}
output {
# stdout { }
elasticsearch {
hosts => ["localhost:9200"]
```
Thank you!!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 10, 2020, 6:41pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/5 "2020-01-10T18:41:03Z")
After the kv, what is the name of the field that you want to trim?
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 10, 2020, 6:44pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/6 "2020-01-10T18:44:37Z")
That's the problem the field is defined as the IP of the device. Being the KV splits the field via "' " and the last record ends "'/\>" it messes up.
I did find out the first entry is defined as ID which is fine. So its only the last one that is a problem.
I hopes this helps
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 10, 2020, 7:12pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/7 "2020-01-10T19:12:11Z")
You are not answering the question. What is the name of the field that you want to trim.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 12, 2020, 10:58pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/8 "2020-01-12T22:58:33Z")
Device is the last field. I am not sure I understand though, because I was on the understanding that the KV defines all the fields. Being KV does not define device because it does not end as expected that field does not exist.
Thank you
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 12, 2020, 11:31pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/9 "2020-01-12T23:31:07Z")
Again you are not answering my question.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 13, 2020, 12:40pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/10 "2020-01-13T12:40:21Z")
The field changes on every log based on the IP address. The value is /\> I showed you the field in the image that is black. its an IP address.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 13, 2020, 12:54pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/11 "2020-01-13T12:54:45Z")
The field is the value that is suppose to be device. so if the last entry is device='10.10.10.10'/\>
the field would be 10.10.10.10 the value is /\>
I think I know what needs to happen
gsub =\> ["message", "/\>", " /\>"] but that does not seem to work.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 14, 2020, 7:47pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/12 "2020-01-14T19:47:10Z")
@Badger
So when I add the
```
mutate {
gsub => ["message", "/>", ""]
```
logs stop processing and when I go to htop there is a java process that is running at 200+% CPU Normal is in the 60% range
So I thought about it and I did
```
mutate {
gsub => ["message", "-", "~"]
```
just as a test and it worked completely fine changing all dashes to tilde
here is a complete message unmodified, not with the ~
``
I am not sure when I mess with the \< /\> the gsub does not work
I have also tried
```
grok {
match => { "message" => "%{IP:device}...$" }}
```
which has not worked but I am guessing my syntax is not correct. This also stops the logging.
Any help would be great
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 14, 2020, 8:17pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/13 "2020-01-14T20:17:46Z")
> [@DWbank](#):
>
```
mutate { gsub => ["message", "$", ""] }
kv { source => "message" field_split => " " value_split => "=" trim_key => " " }
```
works fine for me.
```
"device" => "10.20.5.12",
"fdate" => "2020-01-14 19:36:16",
"src_ip" => "10.20.5.12",
"userdata7" => "VpxLRO",
"userdata4" => "Originator@6876",
"idm_host_src" => "mainserver-1",
"userdata2" => "2099663",
"userdata3" => "FINISH lro",
"proto" => "6",
"data" => "Jan 14 19:36:16 MAINSERVER-1 Vpxa: info vpxa[2099663] [Originator@6876 sub=vpxLro opID=HB-host-124550@45884-6f9ee44f-1b] [VpxLRO] -- FINISH lro-183201 ",
"userdata5" => "vpxLro",
...
```
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 15, 2020, 12:59pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/14 "2020-01-15T12:59:08Z")
I am beginning to think there is something wrong with my system. This conf works but does not get me what I want.
```
input {
tcp {
port => 5142
type => "ossim-events"
codec => json {
charset => "CP1252"
}
}
}
filter {
################### ALIENVAULT OSSIM Logs ###########################
if [type] == "ossim-events" {
kv {
source => "message"
field_split => "' "
value_split => "='"
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
user => username
password => password
}
}
```
This does not. Logs completely stop
```
input {
tcp {
port => 5142
type => "ossim-events"
codec => json {
charset => "CP1252"
}
}
}
filter {
mutate {gsub => ["message", "$", ""] }
################### ALIENVAULT OSSIM Logs ###########################
if [type] == "ossim-events" {
kv {
source => "message"
field_split => " "
value_split => "="
trim_key => " "
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
user => username
password => password
}
}
```
If there is anything else I can look at. I am thinking its not conf related.
I added file and path with rubydebug I have a record it does parse correctly.
---
### Author:  [@DWbank](https://discuss.elastic.co/u/DWbank)
#### Post date: [January 15, 2020, 3:58pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/15 "2020-01-15T15:58:40Z")
@Badger
Thank you that worked, I had to create a new indices and all new policies, I am not sure if that index was corrupt or what.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 12, 2020, 3:58pm UTC](https://discuss.elastic.co/t/removing-leading-character-and-trailing-2-characters/214634/16 "2020-02-12T15:58:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.