# Trim the file name remove last 2 characters and extension using grok

**URL:** <https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631>\
**Category:** Logstash\
**Created:** [June 6, 2019, 4:28pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631 "2019-06-06T16:28:15Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 6, 2019, 4:28pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/1 "2019-06-06T16:28:16Z")

</div>

HI Everyone,

I am trying to push data from s3 to Es using logstash on Ec2.

this is my config file for logstash

input {  
s3 {  
bucket =\> "_"  
access\_key\_id =\> "_"  
secret\_access\_key =\> "\*"  
region =\> "us-east-1"  
codec =\> "json"  
}  
}

filter{  
mutate{  
add\_field =\>{  
"file" =\> "%{[@metadata][s3][key]}"  
}  
}  
}  
output {  
amazon\_es {  
hosts =\> \*  
region =\> "us-east-1"  
aws\_access\_key\_id =\> '_'  
aws\_secret\_access\_key =\> '_'  
index =\> "%{file}"  
template\_name =\> "sqe\_template"  
template\_overwrite =\> "true"  
codec =\> "json"  
}

Now because of "file" =\> "%{[@metadata][s3][key]}"  
i get my index as "filenameins3"

Eg: if I have a file name as log-ecoa-production-index-2019-04-16-17.txt on s3

index comes as log-ecoa-production-index-2019-04-16-17.txt

I want to remove the extension and if required "-17" too and put as index, is there anyway using grok I can achieve this \>  
then index should be log-ecoa-production-index-2019-04-16-17  
or log-ecoa-production-index-2019-04-16 as per requirement

Kindly provide your suggestions.

Thank You,  
Mohit Ruke

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 6, 2019, 4:45pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/2 "2019-06-06T16:45:38Z")

</div>

I would do it with mutate+gsub.

---

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 6, 2019, 4:47pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/3 "2019-06-06T16:47:23Z")

</div>

Yes I researched about it but cant seem to have the right syntax for it  
Can you help ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 6, 2019, 4:54pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/4 "2019-06-06T16:54:13Z")

</div>

```
mutate { gsub => ["message", "\.\w{3}", "", "message", "([0-9]{4}-[0-9]{2}-[0-9]{2})-[0-9]+", "\1" ] }
```

---

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 6, 2019, 5:02pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/5 "2019-06-06T17:02:45Z")

</div>

Thanks Badger,  
I am trying it. Could tell me what script is this line from ```  
mutate { gsub =\> ["message", ".\w{3}", "", "message", "([0-9]{4}-[0-9]{2}-[0-9]{2})-[0-9]+", "\1" ] } ?

like what language does gsub or kibana use ? so for future adjustments I could do it

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 6, 2019, 5:26pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/6 "2019-06-06T17:26:23Z")

</div>

> [@Mohit\_Ruke](#):
>
> like what language does gsub or kibana use ?

gsub uses ruby [regexps](https://ruby-doc.org/core-2.5.3/Regexp.html).

---

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 6, 2019, 5:27pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/7 "2019-06-06T17:27:53Z")

</div>

Thank You, worked!

---

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 7, 2019, 9:21pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/8 "2019-06-07T21:21:02Z")

</div>

> [@Badger](#):
>
> "message", ".\w{3}", "", "message", "([0-9]{4}-[0-9]{2}-[0-9]{2})-[0-9]+", "\1" ]

HI badger !

Now I get the file name as filename-2019-01-32  
How can I get get of the '-32' and set it as index name?

Please help!!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 7, 2019, 11:43pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/9 "2019-06-07T23:43:21Z")

</div>

```
    mutate { add_field => { "someField" => "log-ecoa-production-index-2019-04-16-17.txt" } }
    mutate { gsub => ["someField", "\.\w{3}", "", "message", "([0-9]{4}-[0-9]{2}-[0-9]{2})-[0-9]+", "\1" ] }
    grok { match => ["someField", "-%{INT:anotherField}$"] }

```

will result in

```
"anotherField" => "17",
   "someField" => "log-ecoa-production-index-2019-04-16-17"
```

---

<div class="post-metadata">

**Author:** ![Mohit\_Ruke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohit_ruke/32/46323_2.png) [@Mohit\_Ruke](https://discuss.elastic.co/u/Mohit_Ruke)\
**Post date:** [June 10, 2019, 2:30pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/10 "2019-06-10T14:30:58Z")

</div>

Hi Badger thank you so much for the help. I was able to using this config

filter{

mutate{  
add\_field =\>{  
"file" =\> "%{[@metadata][s3][key]}"  
}  
}  
mutate {  
gsub =\> ["file", ".{3}$", "", "file", "([0-9]{4}-[0-9]{2})-[0-9]+", "\1" ] }  
}

I am getting the required result! But is this the correct way? I dont know the syntax for regex! managed to put this together

Thank You,  
Mohit Ruke

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 8, 2019, 2:30pm UTC](https://discuss.elastic.co/t/trim-the-file-name-remove-last-2-characters-and-extension-using-grok/184631/11 "2019-07-08T14:30:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.