# How to parse logs on startup and merge files per folder?
**URL:** https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291
**Category:** Logstash
**Created:** [September 21, 2017, 8:53am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291 "2017-09-21T08:53:36Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@vrider](https://discuss.elastic.co/u/vrider)
#### Post date: [September 21, 2017, 8:53am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/1 "2017-09-21T08:53:36Z")
Dear Community,
I had troubles with parsing of logs with logstash during startup
Once I add an exta line to log file -\> parsed;
It seems that parsing only is triggerred once change is observer
1. Is there a setting for the input? I've seen http polling, but is there somehting as polling time for a file not http request?
2. I do have number of folders , each folder contains 2 files . Is that possible to parse folder by folder and concatenate fields from these 2 files into a signe json?
Folder o\_1: obs.out, log.out
Folder o\_2: obs.out, log.out
Json formed from timestamp from obs.out and all messages from log.out
So this will be a JSON with "timestamp" :
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 21, 2017, 8:57am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/2 "2017-09-21T08:57:50Z")
> [@vrider](#):
>
> It seems that parsing only is triggerred once change is observer
Yes, by default.
All your questions really depend on what your config looks like.
---
### Author:  [@vrider](https://discuss.elastic.co/u/vrider)
#### Post date: [September 21, 2017, 11:50am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/3 "2017-09-21T11:50:02Z")
My current config at this points looks like this. It is currently only for a single in, single out (don't know so far how to merge two file per folder)
input {
file {
path =\> "C:\logstash-5.6.1\logstash-5.6.1\0\_inputs\netstat\_An.out"
start\_position =\> "beginning" }
}
filter {
grok {
patterns\_dir =\> ["C:\logstash-5.6.1\logstash-5.6.1\0\_patterns"]
match =\> { "message" =\> "%{SPACE} %{WORD:methodtcp} %{SPACE} %{IP:localIp}:%{NUMBER:localport} %{SPACE} %{IP:foreignIp}:%{NUMBER:foreignport} %{SPACE} %{STATUS:status}" }
}
}
output {
file {
path =\> "C:\logstash-5.6.1\logstash-5.6.1\0\_outputs\output.json"
codec =\>
line {
format =\> "out:%{localIp}"
}
}
}
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 21, 2017, 9:23pm UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/4 "2017-09-21T21:23:40Z")
> [@vrider](#):
>
> It is currently only for a single in, single out (don't know so far how to merge two file per folder)
What do you mean by that?
---
### Author:  [@vrider](https://discuss.elastic.co/u/vrider)
#### Post date: [September 27, 2017, 1:26pm UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/5 "2017-09-27T13:26:31Z")
I've figure out how to parse 2 files with 2 file scetion and 2 pths in Input plugin
Is that possible to somehow trigger the reading of file without adding extra lines?
Seems it work only if you add something, like new log records written to a file
Note: we try to parse logs locally from a floder on logstash start
I will certainly appreciate your advise on this
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [September 28, 2017, 4:25am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/6 "2017-09-28T04:25:42Z")
> [@vrider](#):
>
> Is that possible to somehow trigger the reading of file without adding extra lines?
That is the only way.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 26, 2017, 4:25am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291/7 "2017-10-26T04:25:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.