# How to skip some lines from a log file without skipping the complete message

**URL:** https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399
**Category:** Logstash
**Created:** [January 17, 2020, 1:38am UTC](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399 "2020-01-17T01:38:42Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Daniel\_Gonzalez1](https://avatars.discourse-cdn.com/v4/letter/d/57b2e6/32.png) [@Daniel\_Gonzalez1](https://discuss.elastic.co/u/Daniel_Gonzalez1)
#### Post date: [January 17, 2020, 1:38am UTC](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399/1 "2020-01-17T01:38:42Z")

</div>

Hello Everyone!

I'm working with a file that has some lines that I want to skip when they come to logstash.

These are a sample of the log lines:

```
> 15Jan20 11:34:10.39 PRHTGRCE MXP1 RACF CONNECT success for PRHTGRCE: CONNECT C067684
> Jobname + id: PRHTGRCE
> RACF command: CONNECT C067684 AUTHORITY(USE) GROUP(ATRBO01) NOADSP NOAUDITOR NOGRPACC NOOPERATIONS NOSPECIAL OWNER(ATRBO01) RESUME UACC(NONE)
> Name : USER PARA GRUPO CER Instdata : USUARIO PARA LA CONEXION DE GRUPOS RACF REP. OSI
> 
> 15Jan20 11:34:14.61 PRHTGRCE MXP1 RACF ALTUSER success for PRHTGRCE: ALTUSER C110250
> Jobname + id: PRHTGRCE
> RACF command: ALTUSER C110250 NOUAUDIT
> Name : USER PARA GRUPO CER Instdata : USUARIO PARA LA CONEXION DE GRUPOS RACF REP. OSI
> 1zSecure Audit for RACF user events 15Jan20 00:31 to 15Jan20 18:36 list 51
> SMF records for all users with successful commandswq
> 
> Date/time User Sys Description
> 
> 15Jan20 11:34:15.14 PRHTGRCE MXP1 RACF CONNECT success for PRHTGRCE: CONNECT C110250
> Jobname + id: PRHTGRCE
> RACF command: CONNECT C110250 AUTHORITY(USE) GROUP(ATRBO01) NOADSP NOAUDITOR NOGRPACC NOOPERATIONS NOSPECIAL OWNER(ATRBO01) RESUME UACC(NONE)
> Name : USER PARA GRUPO CER Instdata : USUARIO PARA LA CONEXION DE GRUPOS RACF REP. OSI

15Jan20 11:51:28.40 PRHTGRCE MXP1 RACF ALTUSER success for PRHTGRCE: ALTUSER C067734
1zSecure Audit for RACF user events 15Jan20 00:31 to 15Jan20 18:36 list 52
 SMF records for all users with successful commands

 Date/time User Sys Description
   Jobname + id: PRHTGRCE
   RACF command: ALTUSER C067734 NOUAUDIT
   Name : USER PARA GRUPO CER Instdata : USUARIO PARA LA CONEXION DE GRUPOS RACF REP. OSI

```

The thing that I'm looking for is to omit the next lines:

```
1zSecure Audit for RACF user events 15Jan20 00:31 to 15Jan20 18:36 list 51
> SMF records for all users with successful commandswq
> 
> Date/time User Sys Description

```

without skipping the above or the continue of the log data. It suppose that the info that I want to omit are header of a report, this headers comes paste with the logs

Each log line starts with the date

> 15Jan20 11:34:15.14

I used the next config to try to skip all the lines that came with those variables, but it skips the whole message.

```
input {
    file {
        path => ["/home/linux/Downloads/reporte_mf"]
        start_position => "beginning"
        sincedb_path => "/dev/null"
        codec => multiline {
        pattern => "(?<date>\s\d+\d+\w+\d\d\s\d+:\d+:\d+.\d+)"
        negate => true
        what => previous

}
    }
}
# The filter part of this file is commented out to indicate that it is
# optional.
 filter {
      grok{
              match => { "message" => "\s(?<date>\d\d\w+\d\d\s\d+:\d+:\d+.\d+)\s" }

}

      if [message] =~ /(1zSecure.+| SMF records for all users with successful commands| Date\/time.+)/ {
         drop {} }
}

output {

    stdout { codec => rubydebug }
}

```

For example, if in the message the confing finds some of the lines I want to skip, it omit that message, not only the lines I want to skip

Is it possible to erase or omit those lines of the log, without skipping the complete message?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 17, 2020, 3:04pm UTC](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399/2 "2020-01-17T15:04:06Z")

</div>

If there are specific lines you want to omit from events you could use mutate+gsub to remove them.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 14, 2020, 3:04pm UTC](https://discuss.elastic.co/t/how-to-skip-some-lines-from-a-log-file-without-skipping-the-complete-message/215399/3 "2020-02-14T15:04:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.