# Extract special part from access.log file by Logstash

**URL:** https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513
**Category:** Logstash
**Created:** [October 27, 2017, 5:58am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513 "2017-10-27T05:58:26Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![feng\_17](https://avatars.discourse-cdn.com/v4/letter/f/edb3f5/32.png) [@feng\_17](https://discuss.elastic.co/u/feng_17)
#### Post date: [October 27, 2017, 5:58am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513/1 "2017-10-27T05:58:26Z")

</div>

Dear Friends：  
I want to use logstash to extract a special part from access.log file.For instance,to extract all the logs that @timestamp=="2017-10-24T16:41:16+08:00" from the target access.log file to a new file.That is to say “File Input to Filter to File Output”.  
To make this come true,I installed logstash on the host where the access.log file exists.This logstash application is just as a sieve to select the special part of the access.log and make a new file to store them(the special part),that is to say the Input is access.log file,the Output is New file.  
In the first step,I complete the installation of logstash and made a simple test by cmd "bin/logstash -e 'input{stdin{}}output{stdout{codec=\>rubydebug}}' ",this turns out to be all right,it works well.  
At the second step,I made a configuration file called logstash\_core.conf, this is the core part to accomplish my original idea,by the certain plugins used or configured in this file to make the selection come true.  
At the third step,after the configuration,I start the process of logstah by cmd "bin/logstash --verbose -f config/logstash\_core.conf --config.reload.automatic &",but the wanted consequence does not appear.  
I wonder if this(selection) is feasible.If it is ,where is the mistake in my operation?  
The following if the configuration of my logstash application.

#input part  
input{  
file{  
path =\> "/root/10.200.200.54\_access.log" #the path to my access.log file  
type =\> "web\_log"  
codec =\> "json"  
start\_position =\> "beginning"  
}  
}  
#filter part  
filter {  
if [type] == "web-log" {  
mutate {  
gsub =\> ["url-info", "\x", "\\x"]  
}  
mutate {  
gsub =\> ["agent", "\x", "\\x"]  
}  
}  
}  
filter{  
date{  
match =\> ["@timestamp", "yyyy-MM-dd HH:mm:ss","ISO8601"]  
}  
}  
#output part  
output{  
if [@timestamp] == "2017-10-24T16:41:16+08:00" {  
file{  
path =\> "/root/target\_file.log" ##the path to store my new file.  
codec =\> "json"  
}  
}  
}

Look forward to your response.  
Regards.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [October 27, 2017, 6:52am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513/2 "2017-10-27T06:52:07Z")

</div>

- The file input could be tailing the input file since it thinks it has already processed the file. Clearing the sincedb file or setting `sincedb_path` to /dev/null will address that.
- Unless you're running Logstash as root it won't be able to access the input file.
- Are you sure `@timestamp` will ever be equal to "2017-10-24T16:41:16+08:00"? I'm not so sure.

---

<div class="post-metadata">

### Author: ![feng\_17](https://avatars.discourse-cdn.com/v4/letter/f/edb3f5/32.png) [@feng\_17](https://discuss.elastic.co/u/feng_17)
#### Post date: [November 2, 2017, 9:32am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513/3 "2017-11-02T09:32:20Z")

</div>

OK，Thank you at first for your time and energy.

List item

1. I wonder if this is feasible(I mean Extract special part from access.log file by Logstash), make logstash as the filter just to select and save the wanted content of a access.log file? I need a sure answer.Or some constructive advices.
2. About the "sincedb\_path",I will follow your answer and try.
3. Yes,I do it as the system user root,it is OK here.
4. I thought @timestamp as a condition/variability ,so I can filter the special part message I want to save as a new file. This is a try,I wonder if it is OK.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 30, 2017, 9:32am UTC](https://discuss.elastic.co/t/extract-special-part-from-access-log-file-by-logstash/105513/4 "2017-11-30T09:32:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
