# Defining grok patterns
**URL:** https://discuss.elastic.co/t/defining-grok-patterns/40700
**Category:** Logstash
**Created:** [February 2, 2016, 7:19am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700 "2016-02-02T07:19:13Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 2, 2016, 7:19am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/1 "2016-02-02T07:19:13Z")
In logstash, certain patterns are already defined like "COMBINEDAPACHELOG" etc. But what if I want to use my own regular expression? Where do I define it? And how do I use it?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 2, 2016, 7:30am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/2 "2016-02-02T07:30:56Z")
Grok expressions don't need to contain grok patterns like COMBINEDAPACHELOG. In the end it just gets expanded to a normal regular expression anyway. See [https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#\_regular\_expressions](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_regular_expressions).
If you want to define your own grok patterns that you can reuse. See [https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#\_custom\_patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_custom_patterns).
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 2, 2016, 11:02am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/3 "2016-02-02T11:02:23Z")
Can you give me an example where you define the pattern inside grok?
---
### Author:  [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)
#### Post date: [February 2, 2016, 1:12pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/4 "2016-02-02T13:12:16Z")
Hi @Anmol
You have define your grok pattern like this,
grok{
match =\>{"field name" =\> "your own regular expression"}
}
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 2, 2016, 6:46pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/5 "2016-02-02T18:46:57Z")
> Can you give me an example where you define the pattern inside grok?
I'm not sure what you're asking. Do you want to define your own grok pattern, similar to COMBINEDAPACHELOG, and use that in a grok filter? If so the second documentation link I posted contains an example.
---
### Author:  [@anmol](https://discuss.elastic.co/u/anmol)
#### Post date: [February 3, 2016, 6:50am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/6 "2016-02-03T06:50:32Z")
Hey,
I don't know how i got into this conversation but i find it a really
meaningful discussion and hey anmol\_garg i also do think that you should
really check that link
[https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#\_custom\_patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_custom_patterns)
This should solve your problem and if not i'd like to know what exactly you
want.
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 7:20am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/7 "2016-02-03T07:20:03Z")
I got it. Thanks a lot.
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 7:21am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/8 "2016-02-03T07:21:28Z")
> [@Ganesh2303](#):
>
> Hi @AnmolYou have define your grok pattern like this,grok{ match =\>{"field name" =\> "your own regular expression"} }
This doesn't work. I am not sure but using this syntax was giving me an error.
match =\>{"field name" , "your own regular expression"}
This is the syntax that worked.
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 8:53am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/9 "2016-02-03T08:53:10Z")
The main problem with the documentation is the syntax. Like suppose my logs contains lines in 3 types of formats. Then how can i parse each of them using if..else statements? Can you give me an example?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 3, 2016, 9:16am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/10 "2016-02-03T09:16:56Z")
```auto
if some condition {
grok {
match => ["message", "some expression"]
}
} else {
grok {
match => ["message", "some other expression"]
}
}
```
If this doesn't answer your question, please be more specific.
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 9:22am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/11 "2016-02-03T09:22:01Z")
Suppose the lines in my log file either start with a letter or with a "-"
If they start with a letter, I want to parse them differently, and if with a "-", then want to parse them differently
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 3, 2016, 9:28am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/12 "2016-02-03T09:28:52Z")
Then you want a conditional similar to this:
```auto
if [message] =~ /^-/ {
...
}
```
See [https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html) for details and more examples.
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 11:36am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/13 "2016-02-03T11:36:22Z")
Suppose my log is
a=b
x=y
m=n
if I merge them like :
codec =\> multiline{
pattern =\> "^[A-Za-z]"
what =\> "previous"
}
I get a single log.
But what if I want the tags parsed in the way :
a =\> b
x=\> y
m =\> n
The add\_field as well as the add\_tag is not working, because it just combines with the previous log and is only displayed in the message. How do I do that?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 3, 2016, 12:40pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/14 "2016-02-03T12:40:47Z")
I probably don't understand what you mean, but have you looked at the kv filter for parsing key/value pairs? If that doesn't help, please provide examples; what do your events look like now and how would you like them to look?
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 12:45pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/15 "2016-02-03T12:45:24Z")
* * *
UpTime=5 s
HeapMemory=93/245 Mb
SystemMemory=3151/3956 Mb
This is my log file, including the top line with the '-'
This is one log entry. So, while parsing, my input code is :
codec =\> multiline{
pattern =\> "^[A-Za-z]"
what =\> "previous"
}
i.e. Any line starting with a letter, is a part of the previous log. Whenever a line starts with '-', that is a new log entry.
else if [message] =~ /^[A-Za-z]/ {
grok{
match =\> ["message" , "%{WORD:key}=(?[^\n]+)"]
add\_field =\> { "%{key}" =\> "%{value}" }
}
Now this part of the code (the add\_field part), is not being executed. These fields are not shown when I run them into logstash.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [February 3, 2016, 1:07pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/16 "2016-02-03T13:07:40Z")
You said that each line that does not start with '-' belongs to the previous line. I assume that means that each assembled message will start with '-', and not `/^[A-Za-z]/`, which is what you are checking for in your conditional. This should as Magnus pointed out earlier be `/^-/`.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 3, 2016, 2:38pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/17 "2016-02-03T14:38:29Z")
> else if [message] =~ /\[1\]/ {
> grok{
> match =\> ["message" , "%{WORD:key}=(?[^\n]+)"]
> add\_field =\> { "%{key}" =\> "%{value}" }
> }
Where does %{value} come from? Aren't you dealing with the case of multiple key=value tokens in the same string? If so, why not use the kv filter?
* * *
1. A-Za-z
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 3, 2016, 5:34pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/18 "2016-02-03T17:34:27Z")
match =\> ["message" , "%{WORD:key}=(? [^\n]+)"]
Sorry, the code was supposed to be like this.i am mentioning value but it is not being shown here. Anyway..
Ya, it worked with KV. I didn't know about it. Cool
One more thing, can you also guide me through logstash agents? like beats or something. To be more specific, Suppose I have logstash installed on my system, and I want to process logs from another system. How can I do that? Like how can I push logs from some other system, too my system where they can be parsed through logstash?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 3, 2016, 7:49pm UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/19 "2016-02-03T19:49:44Z")
Please read the documentation and start a new thread for any follow-up questions.
[https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-getting-started.html)
---
### Author:  [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)
#### Post date: [February 12, 2016, 4:54am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700/20 "2016-02-12T04:54:02Z")
What are usually the use cases while using the ELK stack for log analysis?
[Next page](https://discuss.elastic.co/t/defining-grok-patterns/40700.md?page=2)