# How to skip second row in log file while processing it through CSV filter

**URL:** <https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314>\
**Category:** Logstash\
**Created:** [July 15, 2020, 2:29pm UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314 "2020-07-15T14:29:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ashish\_kapoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashish_kapoor/32/45171_2.png) [@Ashish\_kapoor](https://discuss.elastic.co/u/Ashish_kapoor)\
**Post date:** [July 15, 2020, 2:29pm UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/1 "2020-07-15T14:29:43Z")

</div>

Hi

I want to skip second line in my logs while processing these logs with CSV filter.

second line of my logs looks like this:

#Remark Values: ComponentType="OAM" ReleaseVersion="11.1.1.9.0"

and logstash conf for processing this file is

```auto
input {
    pipeline { 
        address => OAMAudit
    }  
}
filter {
    mutate {
        gsub => ["message","\\\"","'"]
    }
    csv {
        columns => ["Date", "Time", "Initiator", "EventType", "EventStatus", "MessageText", "AuditUser", "AdditionalInfo", "AdminRoleName", "AgentID", "AgentType", "ApplicationDomainName", "ApplicationName", "AuthenticationMethod", "AuthenticationPolicyID", "AuthenticationSchemeID", "AuthorizationPolicyID", "AuthorizationScheme", "ClientIPAddress", "ConstraintType", "ContextFields", "DataSourceName", "DataSourceType", "DomainName", "ECID", "EventCategory", "FailureCode", "GenericAttribute1", "GenericAttribute2", "GenericAttribute3", "GenericAttribute4", "GenericAttribute5", "HomeInstance", "HostId", "HostIdentifierName", "HostNwaddr", "IdentityDomain", "Impersonator", "InstanceName", "NewAttributes", "NewSettings", "OldAttributes", "OldSettings", "PolicyAdminContext", "PolicyName", "PolicyObjectID", "PolicyType", "ProtectionLevel", "RID", "ReadOnly", "RemoteIP", "RequestID", "Resource", "ResourceHost", "ResourceHostName", "ResourceID", "ResourceOperations", "ResourceTemplateName", "ResourceType", "ResourceURI", "ResponseType", "Roles", "SSOSessionID", "SchemeName", "ServerName", "ServiceIdentifier", "ServiceOperation", "ServiceURI", "SessionCreationTime", "SessionExpirationTime", "SessionID", "SessionLastAccessTime", "SessionLastUpdateTime", "Target", "TargetComponentType", "TenantId", "ThreadId", "TransactionId", "UserDN", "UserID", "UserTenantId"]
        separator => " "
        skip_empty_columns => "true"
        skip_empty_rows => "true"
        skip_header => "true"
    }
     if [Date] == "#Remark" {
        drop { }
    }
    mutate {
      	add_field => {
   		"timestamp2" => "%{Date} %{Time}"
       }
    }
    date {
          match => ["timestamp2", "yyyy-MM-dd HH:mm:ss.SSS", "dd-MM-yyyy HH:mm:ss"]
                    target => "@timestamp"
         }
    mutate {
      remove_field => ["@version","path","host"]
    }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "oam_auditlogs"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 15, 2020, 5:24pm UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/2 "2020-07-15T17:24:23Z")

</div>

> [@Ashish\_kapoor](#):
>
> ```
> if [Date] == "#Remark" { drop { } }
> 
> ```

That looks like a good way to do it. What is your question?

---

<div class="post-metadata">

**Author:** ![Ashish\_kapoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashish_kapoor/32/45171_2.png) [@Ashish\_kapoor](https://discuss.elastic.co/u/Ashish_kapoor)\
**Post date:** [July 15, 2020, 6:55pm UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/3 "2020-07-15T18:55:54Z")

</div>

Hi

I am not able to skip this line with below code

```auto
if [Date] == "#Remark" {
        drop { }
    }

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 15, 2020, 7:03pm UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/4 "2020-07-15T19:03:13Z")

</div>

You could try

```
if [message] =~ /^#Remark/ { drop {} }

```

_before_ the csv filter.

Are the fields in the CSV wrapped in quotes?

---

<div class="post-metadata">

**Author:** ![Ashish\_kapoor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashish_kapoor/32/45171_2.png) [@Ashish\_kapoor](https://discuss.elastic.co/u/Ashish_kapoor)\
**Post date:** [July 17, 2020, 8:49am UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/5 "2020-07-17T08:49:29Z")

</div>

Thanks for help. I able to resolve this issue, But I need to delete header and second line in message Manually.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 14, 2020, 8:49am UTC](https://discuss.elastic.co/t/how-to-skip-second-row-in-log-file-while-processing-it-through-csv-filter/241314/6 "2020-08-14T08:49:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.