Grok Custom pattern file

Elastic Stack Logstash
1

Hi guys,

I'm starting in logstash's world and i'm having a problem to set more than one custom grok pattern into a "patern_dir"

I've created a file into my directory "/etc/logstash/pattern" with the follow information

#GROK CUSTOM PATTERNS INFORMATION CONTENTS
FILE_VERSION \w+\d{6}(?=_)
BU_ID \d{3}(?=\.)

I want to know if it needs some separator between both patterns.

thank for any help !

2

A single newline character is the only separator you need. Your file looks fine.

3

I'm trying to take a part name of the following source path

path => /elasticsearch/logstash/example/example201610_007.csv"

this is in a input file.
i want to use the grok filter to set my custom patterns on the index name.

filter {
    grok{
             patterns_dir => ["/elasticsearch/logstash/example/patterns"]
             match =>{"file_version" => "%{FILE_VERSION:%{path}}" }
             match =>{"bu_id" => "%{BU_ID:%{path}}" }
         }
    }

But I'm not getting the expected result when the index is created.

Thanks for your help.

4 
         match =>{"file_version" => "%{FILE_VERSION:%{path}}" }
        match =>{"bu_id" => "%{BU_ID:%{path}}" }

What are you trying to do here?

5

I was trying to do the field path get the pattern syntax before.

But i think i've had a mistake.

i'll put here my config file for you to see.

input {
  file {
    path => "/elasticsearch/logstash/example/example201610_007.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ";"
   columns => ["field1","field2","field3","field4"]
  }

  mutate {
    rename => { "field2" => "productId" }
    rename => { "field3" => "productComplementaryId" }
    rename => { "field4" => "productComplementaryName" }
  }
 
   grok{
    
    patterns_dir => ["/elasticsearch/logstash/example/patterns"]
    match =>{"%{path}" => "%{FILE_VERSION:file_version}" }
    match =>{"%{path}" => "%{BU_ID:bu_id}" }
    
  }

}
output {
    stdout { codec => rubydebug { metadata => true } }
    elasticsearch {
        index => "gabriel_%{file_version}"
        document_type => "complementary%{bu_id}"
        document_id => "%{productId}%{productComplementaryId}"
        hosts => "${LOGSTASH_ES_HOSTS}"
  
     }
}

I want to take a part of path field.

1 Like
6

What you probably want is something like this:

grok {
  match => {
    "path" => "/%{FILE_VERSION:version}_%{BU_ID:id}\.csv$"
  }
  patterns_dir => ["/elasticsearch/logstash/example/patterns"]
}
1 Like
7

Thanks a lot , it was what i needed.

It works perfetly !

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.