# Parsing in logstash using (grok,prune,multiline filter}

**URL:** <https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125>\
**Category:** Logstash\
**Created:** [January 11, 2018, 3:49pm UTC](https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125 "2018-01-11T15:49:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cilzzz](https://avatars.discourse-cdn.com/v4/letter/c/c67d28/32.png) [@cilzzz](https://discuss.elastic.co/u/cilzzz)\
**Post date:** [January 11, 2018, 3:49pm UTC](https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125/1 "2018-01-11T15:49:49Z")

</div>

Hello everyone,  
i have an export of a script below that i need to parse through logstash.

```
Job Name : save_start_config-sftp Job Status: Success (0)
Schedule Name : daily_save_start_config User Name : service-mainframe
Completion time: Mon Jan 8 06:00:21 2018
--------------------------------- Job Output ---------------------------------
`run-script bootflash:/save_start_config_sftp.vsh`
`copy running-config startup-config
`

[#] 1%
[#] 2%
[##] 3%
[##] 4%
[###] 5%
[###] 6%
[###] 7%
[####] 8%
[#####] 10%
[#####] 11%
[#####] 12%
[######] 13%
[######] 14%
[#######] 15%
[#######] 16%
[#######] 17%

```

* * *

* * *

* * *

```
[########################################] 98%
[########################################] 100%
Copy complete.
`copy startup-config sftp://service-mainframe@10.155.158.186./home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt
`
Connected to 10.155.158.186.
sftp> 
sftp> put /var/tmp/vsh/SANMF-DAL-A-startup-config /home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt 
Uploading /var/tmp/vsh/SANMF-DAL-A-startup-config to /home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt
sftp> exit 

Copy complete.

```

what i need is to drop everything till the (first) copy complete so this the part that i need from the log

```
copy startup-config sftp://service-mainframe@10.155.158.186/home/service-mainframe/dir_CCH-CHA-B/conf_WWEF-PAL-A_2018-01-10-06.00.16.txt
Connected to 10.155.158.186
sftp>
sftp> put /var/tmp/vsh/CCH-CHA-B-startup-config /home/service-mainframe/dir_CCH-CHA-B/conf_WWEF-PAL-A_2018-01-10-06.00.16.txt
Uploading /var/tmp/vsh/CCH-CHA-B-startup-config to /home/service-maiframe/dir_CCH-CHA-B/conf_WWEF-PAL-A_2018-01-10-06.00.16.txt
sftp> exit
Copy complete.

```

My grok patterns are:

```
%{WORD:action1} %{WORD}-%{WORD} %{WORD:protocol}://%{USER:utilisateur}@%{IP:clientip}/%{GREEDYDATA:repertoire}
%{GREEDYDATA} %{GREEDYDATA} %{IP}
%{GREEDYDATA}
%{WORD}> %{WORD:action2} %{GREEDYDATA:source} %{GREEDYDATA:destination}
%{WORD:action3} %{GREEDYDATA}
%{WORD}> %{WORD:action4}
%{GREEDYDATA:Status}

```

and the results are:

```
{
  "action1": [
    "copy"
  ],
  "protocol": [
    "sftp"
  ],
  "utilisateur": [
    "service-mainframe"
  ],
  "clientip": [
    "10.155.158.186"
  ],
  "repertoire": [
    "home/service-mainframe/dir_CCH-CHA-B/conf_WWEF-PAL-A_2018-01-10-06.00.16.txt"
  ],
  "action2": [
    "put"
  ],
  "source": [
    " /var/tmp/vsh/CCH-CHA-B-startup-config "
  ],
  "destination": [
    "/home/service-mainframe/dir_CCH-CHA-B/conf_WWEF-PAL-A_2018-01-10-06.00.16.txt"
  ],
  "action3": [
    "Uploading"
  ],
  "action4": [
    "exit"
  ],
  "Status": [
    "Copy complete."
  ]
}

```

how can i do this in logstash 2.4 what filter should i use other than Grok (prune or multiline could help?}  
i am just a beginner in logstash what is the easiest way to parse this file through logstash, what i need do is:

> 1- Get rid of the first part of the log file (prune)  
> 2-change the multine into a line (multiline filter)  
> 3-write a grok patterns (grok filter)  
> 4-assign each pattern as a field to make a dashboard on kibana

thank you !

---

<div class="post-metadata">

**Author:** ![Sajeew.Ganesh](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@Sajeew.Ganesh](https://discuss.elastic.co/u/Sajeew.Ganesh)\
**Post date:** [January 12, 2018, 10:02am UTC](https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125/2 "2018-01-12T10:02:24Z")

</div>

Did you try with the grok patterns you have written above?

---

<div class="post-metadata">

**Author:** ![cilzzz](https://avatars.discourse-cdn.com/v4/letter/c/c67d28/32.png) [@cilzzz](https://discuss.elastic.co/u/cilzzz)\
**Post date:** [January 12, 2018, 10:33am UTC](https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125/3 "2018-01-12T10:33:09Z")

</div>

the grok patterns worked on the grok debugger site but at first i need to skip all the lines and keep only this part below to apply my grok patterns

```
`copy startup-config sftp://service-mainframe@10.155.158.186./home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt
`
Connected to 10.155.158.186.
sftp> 
sftp> put /var/tmp/vsh/SANMF-DAL-A-startup-config /home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt 
Uploading /var/tmp/vsh/SANMF-DAL-A-startup-config to /home/service-mainframe/conf_WWF_CH_C_2018-01-08-06.00.16.txt
sftp> exit 

Copy complete.
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 9, 2018, 10:33am UTC](https://discuss.elastic.co/t/parsing-in-logstash-using-grok-prune-multiline-filter/115125/4 "2018-02-09T10:33:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.