# Extracting path fragments from \`log.file.path\` field

**URL:** <https://discuss.elastic.co/t/extracting-path-fragments-from-log-file-path-field/359726>\
**Category:** Logstash\
**Created:** [May 17, 2024, 3:33pm UTC](https://discuss.elastic.co/t/extracting-path-fragments-from-log-file-path-field/359726 "2024-05-17T15:33:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![neesa](https://avatars.discourse-cdn.com/v4/letter/n/90ced4/32.png) [@neesa](https://discuss.elastic.co/u/neesa)\
**Post date:** [May 17, 2024, 3:33pm UTC](https://discuss.elastic.co/t/extracting-path-fragments-from-log-file-path-field/359726/1 "2024-05-17T15:33:29Z")

</div>

Hi,

I'm trying to extract two strings from the metadata `log.file.path` field, and create one or two fields, depending on whether the first of these fields has the appropriate value.

Sample `log.file.path` field content of event are as follows:

- /usr/share/logstash/ingest\_data/MY-APP-B/20240517/MY-APP-B-JOBNAMEX-28599000-fqlg6-12.log`
- /usr/share/logstash/ingest\_data/MY-APP-A/20240517/MY-APP-A-6b9bbc6864-plgns-16.log
- /usr/share/logstash/ingest\_data/MY-APP-B/20240517/MY-APP-B-JOBNAMEY-145-manual-w68qx-13.log
- /usr/share/logstash/ingest\_data/MY-APP-C/20240517/MY-APP-C-vsgxg-16.log

The text strings I want to extract are:

- `MY-APP-<letter>` (e.g. `MY-APP-A` or `MY-APP-B`) and save it in the `applicationName` field
- `JOBNAME<letter>` (e.g. `JOBNAMEX` or `JOBNAMEY`) and save it in the `jobName` field, but only if `applicationName` is `MY-APP-B`.

In the logstash configuration file I created a filter:

```auto
  mutate { copy => { "[log][file][path]" => "[@metadata][path]" } }
  mutate { split => { "[@metadata][path]" => "/" } }
  mutate { add_field => { "applicationName" => "%{[@metadata][path][5]}" } }
  if [applicationName] == "MY-APP-B" {
    mutate { copy => { "[log][file][path]" => "[@metadata][jobNameFull]" } }
    mutate { split => { "[@metadata][jobNameFull]" => "-" } }
    mutate { add_field => { "jobName" => "%{[@metadata][jobNameFull][3]}" } }
  }

```

It basically works, but it is not resistant to changing the location of log files (I suppose that `/usr/share/logstash` part could be changed in future). Moreover, I think the `{ split => { "[@metadata][jobNameFull]" => "-" }` expression is very fragile because it relies on any `-` appearing anywhere in the `log.file.path` field.

Is there a way to make this filter more universal and safe?

1. Is it possible to create a filter that will assign a value to the `applicationName` field from what is between the strings `/ingest_data/` and the next `/`? Or maybe would be better string between `/` and `/YYYYMMDD` (20240517 in my case). Not sure which is the best...
2. Is it possible to create a filter that will assign a value to the `jobName` field from what is between `MY-APP-B` (which contains `-`) and the next character `-`?

Maybe some regex? But I'm not sure, how to use it with conjunction with [field references](https://www.elastic.co/guide/en/logstash/8.13/event-dependent-configuration.html).

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 17, 2024, 4:17pm UTC](https://discuss.elastic.co/t/extracting-path-fragments-from-log-file-path-field/359726/2 "2024-05-17T16:17:56Z")

</div>

I would keep it simple and just use grok with regexps that just match the part of the path that you want

```
    grok { "match" => { "[log][file][path]" => "/(?<appName>[A-Za-z\-]+)/%{NUMBER:date}/" } }
    if [appName] == "MY-APP-B" {
        grok { "match" => { "[log][file][path]" => "/MY-APP-B-(?<jobName>JOBNAME\w)-" } }
    }

```

---

<div class="post-metadata">

**Author:** ![neesa](https://avatars.discourse-cdn.com/v4/letter/n/90ced4/32.png) [@neesa](https://discuss.elastic.co/u/neesa)\
**Post date:** [May 22, 2024, 8:11am UTC](https://discuss.elastic.co/t/extracting-path-fragments-from-log-file-path-field/359726/3 "2024-05-22T08:11:30Z")

</div>

Thank you @Badger, it works.

The simplicity of your solution is brilliant!

Best regards.