# Make a field "less specific"

**URL:** https://discuss.elastic.co/t/make-a-field-less-specific/72473
**Category:** Logstash
**Created:** [January 23, 2017, 11:29am UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473 "2017-01-23T11:29:53Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 23, 2017, 11:29am UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/1 "2017-01-23T11:29:53Z")

</div>

Hi,

I am new in Logstash, I could perform my first log analysis without any problem but I'm facing a situation I have not find a solution for.

So in my logs I have a path that I manage to isolate easily, but here is the thing, this path may include some ID's that I don't need.  
For example I have this line :  
/Manager/Playlists/ **48d38243-1def-45a** /Entries/ **0f875fde-37ea-410c** /Clip/Export

So I would like to make those ID in bold all the same, as I care more about the kind of path, more than the specific element I deal with.  
So far I managed to get all the fields seperately and my idea was to reconstruct a simplified field, by replacing the IDs to a static string in a mutate, and recreating a new field that would become for the previous example :  
/Manager/Playlists/idString/Entries/idString/Clip/Export

But at this moment the problem is that the paths I deal with don't all have the same pattern, so the IDs are not at always the same positions.  
And unless I'm mistaken (?) we can't create a new field including optional fields.

So I was wondering if it's possible to change the field on the fly while we parse (within the grok). Or maybe there is another solution I haven't think about.

Thanks to all of you, and hope I have been clear enough in my explanation

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 23, 2017, 12:08pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/2 "2017-01-23T12:08:53Z")

</div>

Would the mutate filter's gsub option help?

---

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 23, 2017, 12:17pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/3 "2017-01-23T12:17:08Z")

</div>

thanks for your input, I indeed didn't check this possibility  
can we apply it in the whole field like this for example :  
"fieldname", "\*", "idString"

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 23, 2017, 12:19pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/4 "2017-01-23T12:19:28Z")

</div>

You mean replace the field value completely? Sure, but the mutate filter's replace option is more efficient for that.

---

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 23, 2017, 12:27pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/5 "2017-01-23T12:27:00Z")

</div>

oh yes but actually the same problem will occur I guess, it will change for the field taken individually but not within the path right?  
say I have those fields :  
/Manager/Playlists/ **48d38243-1def-45a** /Entries/ **0f875fde-37ea-410c** /Clip/Export =\> myPath  
Manager =\> part1  
Playlist =\> part2  
**48d38243-1def-45a** =\> id  
Entries =\> part4  
**0f875fde-37ea-410c** =\> id  
Clip =\> part6  
Export =\> part7

If I use replace or gsub on id it won't replace within myPath, is there a solution to "update" myPath with the new value?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 23, 2017, 12:32pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/6 "2017-01-23T12:32:21Z")

</div>

Yes, use gsub.

---

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 23, 2017, 12:33pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/7 "2017-01-23T12:33:57Z")

</div>

ok I will try.

thank you very much

---

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 23, 2017, 1:52pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/8 "2017-01-23T13:52:50Z")

</div>

after trying the problem remain.  
I would like the field to be recursively changed in myPath  
or maybe I miss something

I might have to write a ruby piece of code

---

<div class="post-metadata">

### Author: ![migou](https://avatars.discourse-cdn.com/v4/letter/m/67e7ee/32.png) [@migou](https://discuss.elastic.co/u/migou)
#### Post date: [January 24, 2017, 1:53pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/9 "2017-01-24T13:53:33Z")

</div>

ok I managed to do what I wanted :  
I added all the new\_field (part1, part2 etc) beforehand with empty string as value.  
then I parsed my path as I wanted (I also managed to write a general match with optional fields), overwriting the fields previously created, when a new value is found.

Then I wrote a Ruby code to change all the IDs that are not empty string to "idString"  
then I reconstruct all the paths with all the fields (they all exists though some are just an empty string so no need to deal with optional)

Thanks Magnus for your inputs anyway 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 21, 2017, 1:53pm UTC](https://discuss.elastic.co/t/make-a-field-less-specific/72473/10 "2017-02-21T13:53:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
