# Extracting filename from existing field (Source)?
**URL:** https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521
**Category:** Logstash
**Created:** [November 6, 2018, 10:00am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521 "2018-11-06T10:00:49Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 10:00am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/1 "2018-11-06T10:00:49Z")
how can i extract directory name from the below ?
source path = "/var/log/companyname/servicename/jjds/smtgxyz/smtgabc.log"
from the above i have to get "servicename" as an another field. can someone please provide the example use case?
---
### Author:  [@Jenni](https://discuss.elastic.co/u/Jenni)
#### Post date: [November 6, 2018, 10:24am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/2 "2018-11-06T10:24:58Z")
Like this?
`/var/log/.*?/(?.*?)/`
```
{
"logfolder": [
[
"servicename"
]
]
}
```
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 10:36am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/3 "2018-11-06T10:36:43Z")
exactly
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 10:39am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/4 "2018-11-06T10:39:04Z")
how do i extract the same from existing field "source"?
---
### Author:  [@Jenni](https://discuss.elastic.co/u/Jenni)
#### Post date: [November 6, 2018, 10:41am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/5 "2018-11-06T10:41:46Z")
Oh sorry. I forgot to mention grok.
```
filter {
grok {
match => { "source" => "/var/log/.*?/(?.*?)/" }
}
}
```
Please don't send me private messages. I would have answered your question anyway.
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 10:49am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/6 "2018-11-06T10:49:00Z")
[2018-11-06T10:47:46,868][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\, :backtrace=\>["org/jruby/RubyRegexp.java:928:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in`compile'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:281:in `block in register'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1343:in`each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:270:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:241:in`register\_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in`maybe\_setup\_out\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:262:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:199:in`run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:159:in `block in start'"], :thread=\>"#\"}
[2018-11-06T10:47:46,890][ERROR][logstash.agent] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create\, action\_result: false", :backtrace=\>nil}
getting below error after adding
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 10:54am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/7 "2018-11-06T10:54:09Z")
[2018-11-06T10:50:11,840][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[//localhost:9200](https://localhost:9200)"]}
[2018-11-06T10:50:12,279][ERROR][logstash.pipeline] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\, @filter=\["/etc/logstash/conf.d/patterns"], match=\>{"message"=\>"%{LOGLEVEL:log\_lvl}\\s+\\[%{TIMESTAMP\_ISO8601:time\_stamp}\\]\\s+\\[%{REQID:req\_id}\\]\\s+%{JAVACLASS:class\_name}:\\s+%{JAVALOGMESSAGE:log\_msg}\\s+%{services:services}", "source"=\>"/var/log/.?/(?.?)/"}, id=\>"02ebfe22ba3c411cc37239b7f5cbce01ca59f2a2682af87f0dd1132b12f453ae", enable\_metric=\>true, periodic\_flush=\>false, patterns\_files\_glob=\>"\*", break\_on\_match=\>true, named\_captures\_only=\>true, keep\_empty\_captures=\>false, tag\_on\_failure=\>["\_grokparsefailure"], timeout\_millis=\>30000, tag\_on\_timeout=\>"\_groktimeout"\>\>", :error=\>"undefined group option: /\/var\/log\/.?\/(?.?)\//m", :thread=\>"#\"}
[2018-11-06T10:50:12,288][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\, :backtrace=\>["org/jruby/RubyRegexp.java:928:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:127:in`compile'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:281:in `block in register'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1343:in`each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:270:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:241:in`register\_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in`maybe\_setup\_out\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:262:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:199:in`run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:159:in `block in start'"], :thread=\>"#\"}
[2018-11-06T10:50:12,318][ERROR][logstash.agent] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create\, action\_result: false", :backtrace=\>nil}
---
### Author:  [@Jenni](https://discuss.elastic.co/u/Jenni)
#### Post date: [November 6, 2018, 10:56am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/8 "2018-11-06T10:56:32Z")
I don't really know what's wrong.
My little test works fine:
```
input {
stdin {}
}
filter {
grok {
match => { "message" => "/var/log/.*?/(?.*?)/" }
}
}
output {
stdout { codec => rubydebug }
}
```
> The stdin plugin is now waiting for input:
> /var/log/companyname/servicename/jjds/smtgxyz/smtgabc.log
> {
> "@version" =\> "1",
> "message" =\> "/var/log/companyname/servicename/jjds/smtgxyz/smtgabc.log",
> "@timestamp" =\> 2018-11-06T10:54:51.182Z,
> "logfolder" =\> "servicename",
> "host" =\> "elasticsearch-vm"
> }
What does your configuration look like?
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 11:06am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/9 "2018-11-06T11:06:28Z")
input {
beats {
port =\> 5044
host =\> "0.0.0.0"
}
}
filter {
grok {
patterns\_dir =\> ["/etc/logstash/conf.d/patterns"]
match =\> { "message" =\> "%{LOGLEVEL:log\_lvl}\s+[%{TIMESTAMP\_ISO8601:time\_stamp}]\s+[%{REQID:req\_id}]\s+%{JAVACLASS:class\_name}:\s+%{JAVALOGMESSAGE:log\_msg}\s+%{services:services}"}
match =\> { "servicename" =\> "/var/log/._?/(?._?)/" }
}
mutate {
remove\_field =\> ["message"]
#remove\_field =\> ["beat"]
remove\_field =\> ["[host]"]
}
mutate {
add\_field =\> {
"host" =\> "%{[beat][hostname]}
}
}
}
output {
elasticsearch {
user =\> "elastic"
password =\> "changeme"
hosts =\> "localhost:9200"
manage\_template =\> false
index =\> "logstash-%{+YYYY.MM.dd}"
document\_type =\> "%{[@metadata][type]}"
}
}
this isn't showing any data regarding "logfolder"!!
---
### Author:  [@Jenni](https://discuss.elastic.co/u/Jenni)
#### Post date: [November 6, 2018, 11:19am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/10 "2018-11-06T11:19:40Z")
For a start you shouldn't define the same option twice in one plugin. That won't work.
Furthermore: Is that really your config?
> [@roly](#):
>
> match =\> { "servicename" =\> "/var/log/. _?/(?._ ?)/" }
That doesn't look like my suggestion at all...
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 11:24am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/11 "2018-11-06T11:24:16Z")
input {
beats {
port =\> 5044
host =\> "0.0.0.0"
}
}
filter {
grok {
patterns\_dir =\> ["/etc/logstash/conf.d/patterns"]
match =\> { "message" =\> "%{LOGLEVEL:log\_lvl}\s+[%{TIMESTAMP\_ISO8601:time\_stamp}]\s+[%{REQID:req\_id}]\s+%{JAVACLASS:class\_name}:\s+%{JAVALOGMESSAGE:log\_msg}\s+%{services:services}"}
```
match => { "source" => "/var/log/.*?/(?.*?)/" }
```
}
mutate {
remove\_field =\> ["message"]
#remove\_field =\> ["beat"]
remove\_field =\> ["[host]"]
}
mutate {
add\_field =\> {
"host" =\> "%{[beat][hostname]}"
"servicename" =\> "%{[logfolder]}"
}
}
}
output {
elasticsearch {
user =\> "elastic"
password =\> "changeme"
hosts =\> "localhost:9200"
manage\_template =\> false
index =\> "logstash-%{+YYYY.MM.dd}"
document\_type =\> "%{[@metadata][type]}"
}
}
this is how my config file is!!
---
### Author:  [@roly](https://discuss.elastic.co/u/roly)
#### Post date: [November 6, 2018, 11:27am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/12 "2018-11-06T11:27:27Z")
basically we are already getting our data to elk. But we also wanted our service name from the source path. so we thought of taking it from existing filed "source" using regex.
we want to get the information from existing logstash's field which is "source" field!!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 4, 2018, 11:27am UTC](https://discuss.elastic.co/t/extracting-filename-from-existing-field-source/155521/13 "2018-12-04T11:27:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.