# Losgstash error starting with Expected one of #, {, ,, \] at line XX
**URL:** https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072
**Category:** Logstash
**Created:** [April 30, 2019, 11:25am UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072 "2019-04-30T11:25:45Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@franco.federico](https://discuss.elastic.co/u/franco.federico)
#### Post date: [April 30, 2019, 11:25am UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/1 "2019-04-30T11:25:46Z")
I create a grok filter in order to capture this information with logstash from a syslog message
The message of Syslog is like this
"Logon Attempt";"4624";"Audit Success";"NetApp-Security-Auditing";"CIFS";"Security";"ITF11000/svm-M-cifs";"2/1/2019 4:38:41";"10.8.38.80";"61035";"S-1-5-21-2976626676-2712618565-3739747982-140608";"IT01896";"false";"GRTGROUP";"3";"KRB5";"";"";""
Using grok debug site I found that the following grok run
-|"(%{GREEDYDATA:eventname})");(-|"(%{NUMBER:event\_id})");(-|"(%{GREEDYDATA:audit\_result})");(-|"(%{GREEDYDATA:provider})");(-|"(%{GREEDYDATA:source\_netapp})");(-|"(%{GREEDYDATA:security})");(-|"(%{GREEDYDATA:computer})");(-|"(%{DATESTAMP:created\_date})");(-|"(%{IP:ip})");(-|"(%{GREEDYDATA:ip\_port})");(-|"(%{GREEDYDATA:target\_userid})");(-|"(%{USERNAME:username})");(-|"(%{GREEDYDATA:target\_user\_is\_local})");(-|"(%{GREEDYDATA:domain})");(-|"(%{GREEDYDATA:logon\_type})");(-|"(%{GREEDYDATA:authentication\_package\_name})");(-|"(%{GREEDYDATA:error\_code})");(-|"(%{GREEDYDATA:error\_code\_reason})");(-|"(%{GREEDYDATA:error\_code\_reason\_string})")
And I found the data.
So I create a filter in order to extract information and then put in ES
My filter is the following
filter {
if [message] =~ /^\s\*Safed[/ {
mutate {
replace =\> ["source", "Safed"]
}
}
if [source] == "Safed"{
# Safed without stripped syslog-tag "Safed[...][...]"
grok {
patterns\_dir =\> "/neteye/shared/logstash/conf/pattern.d"
match =\> ["message", "(-|"(%{GREEDYDATA:eventname})");(-|"(%{NUMBER:event\_id})");(-|"(%{GREEDYDATA:audit\_result})");(-|"(%{GREEDYDATA:provider})");(-|"(%{GREEDYDATA:source\_netapp})");(-|"(%{GREEDYDATA:security})");(-|"(%{GREEDYDATA:computer})");(-|"(%{DATESTAMP:created\_date})");(-|"(%{IP:ip})");(-|"(%{GREEDYDATA:ip\_port})");(-|"(%{GREEDYDATA:target\_userid})");(-|"(%{USERNAME:username})");(-|"(%{GREEDYDATA:target\_user\_is\_local})");(-|"(%{GREEDYDATA:domain})");(-|"(%{GREEDYDATA:logon\_type})");(-|"(%{GREEDYDATA:authentication\_package\_name})");(-|"(%{GREEDYDATA:error\_code})");(-|"(%{GREEDYDATA:error\_code\_reason})");(-|"(%{GREEDYDATA:error\_code\_reason\_string})")"]
overwrite =\> ["message"]
remove\_tag =\> "\_grokparsefailure"
break\_on\_match =\> false
}
mutate {
convert =\> { "event\_id" =\> "integer" }
}
if [event\_id] in [4634, 4647] {
mutate {
add\_field =\> ["audit\_type", "LOGOFF"]
add\_tag =\> "AUDIT\_LOGOFF"
}
}
if [event\_id] in [4624, 4647] {
mutate {
add\_field =\> ["audit\_type", "LOGON"]
add\_tag =\> "AUDIT\_LOGON"
}
}
if [event\_id] in [4625, 4647] {
mutate {
add\_field =\> ["audit\_type", "FAILURE"]
add\_tag =\> "AUDIT\_FAILURE"
}
}
}
}
And when the logstash restart I found the error Expected one of #, {, ,, ] at line XX. I show the entire error.
^[[2019-04-30T13:12:16,552][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #, {, ,, ] at line 59, column 34 (byte 1239) after filter {\n if [message] =~ /^\s\*Safed\[/ {\n mutate {\n replace =\> ["source", "Safed"]\n }\n }\n if [source] == "Safed"{\n # Safed without stripped syslog-tag "Safed[...][...]"\n grok {\n patterns\_dir =\> "/neteye/shared/logstash/conf/pattern.d"\n match =\> ["message", "(-|"", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in`compile\_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in`map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in`block in converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in`converge\_state\_and\_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:105:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in`interval'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:94:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in`block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
I'm searching in the logstash.conf and I found that the point is here
```
match => ["message", "(-|"(%{GREEDYDATA:eventname})");(-|"(%{NUMBER:event_id})");(-|"(%{GREEDYDATA:audit_result})");(-|"(%{GREEDYDATA:provider})");(-|"(%{GREEDYDATA:source_netapp})");(-|"(%{GREEDYDATA:security})");(-|"(%{GREEDYDATA:computer})");(-|"(%{DATESTAMP:created_date})");(-|"(%{IP:ip})");(-|"(%{GREEDYDATA:ip_port})");(-|"(%{GREEDYDATA:target_userid})");(-|"(%{USERNAME:username})");(-|"(%{GREEDYDATA:target_user_is_local})");(-|"(%{GREEDYDATA:domain})");(-|"(%{GREEDYDATA:logon_type})");(-|"(%{GREEDYDATA:authentication_package_name})");(-|"(%{GREEDYDATA:error_code})");(-|"(%{GREEDYDATA:error_code_reason})");(-|"(%{GREEDYDATA:error_code_reason_string})")"]
```
Why? In the grok debug it's ok.
Thank you
franco
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 1:51pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/2 "2019-04-30T13:51:49Z")
> [@franco.federico](#):
>
> match =\> ["message", "(-|"(%{GREEDYDATA:eventname})")[...]
You have double quotes in your pattern that is surrounded by double quotes. You can either escape them all with backslash, or you can replace the double quotes around your pattern with single quotes.
---
### Author:  [@franco.federico](https://discuss.elastic.co/u/franco.federico)
#### Post date: [April 30, 2019, 1:57pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/3 "2019-04-30T13:57:05Z")
Thank you for reply. I solved the problem with the single quote with this string
match =\> ["message", '(-|"(%{GREEDYDATA:eventname})");(-|"(%{NUMBER:event\_id})");(-|"(%{GREEDYDATA:audit\_result})");(-|"(%{GREEDYDATA:provider})");(-|"(%{GREEDYDATA:source\_netapp})");(-|"(%{GREEDYDATA:security})");(-|"(%{GREEDYDATA:computer})");(-|"(%{DATESTAMP:created\_date})");(-|"(%{IP:ip})");(-|"(%{GREEDYDATA:ip\_port})");(-|"(%{GREEDYDATA:target\_userid})");(-|"(%{USERNAME:username})");(-|"(%{GREEDYDATA:target\_user\_is\_local})");(-|"(%{GREEDYDATA:domain})");(-|"(%{GREEDYDATA:logon\_type})");(-|"(%{GREEDYDATA:authentication\_package\_name})");(-|"(%{GREEDYDATA:error\_code})");(-|"(%{GREEDYDATA:error\_code\_reason})");(-|"(%{GREEDYDATA:error\_code\_reason\_string})")']
Now I have another problem I'm trying to use date filter
I change my filter file with this
filter {
if [message] =~ /^\s\*Safed[/ {
mutate {
replace =\> ["source", "Safed"]
}
}
if [source] == "Safed"{
# Safed without stripped syslog-tag "Safed[...][...]"
grok {
patterns\_dir =\> "/neteye/shared/logstash/conf/pattern.d"
match =\> ["message", '(-|"(%{GREEDYDATA:eventname})");(-|"(%{NUMBER:event\_id})");(-|"(%{GREEDYDATA:audit\_result})");(-|"(%{GREEDYDATA:provider})");(-|"(%{GREEDYDATA:source\_netapp})");(-|"(%{GREEDYDATA:security})");(-|"(%{GREEDYDATA:computer})");(-|"(%{DATESTAMP:created\_date})");(-|"(%{IP:ip})");(-|"(%{GREEDYDATA:ip\_port})");(-|"(%{GREEDYDATA:target\_userid})");(-|"(%{USERNAME:username})");(-|"(%{GREEDYDATA:target\_user\_is\_local})");(-|"(%{GREEDYDATA:domain})");(-|"(%{GREEDYDATA:logon\_type})");(-|"(%{GREEDYDATA:authentication\_package\_name})");(-|"(%{GREEDYDATA:error\_code})");(-|"(%{GREEDYDATA:error\_code\_reason})");(-|"(%{GREEDYDATA:error\_code\_reason\_string})")']
overwrite =\> ["message"]
remove\_tag =\> "\_grokparsefailure"
break\_on\_match =\> false
}
mutate {
convert =\> { "event\_id" =\> "integer" }
}
if [event\_id] in [4634, 4647] {
mutate {
add\_field =\> ["audit\_type", "LOGOFF"]
add\_tag =\> "AUDIT\_LOGOFF"
}
}
if [event\_id] in [4624, 4647] {
mutate {
add\_field =\> ["audit\_type", "LOGON"]
add\_tag =\> "AUDIT\_LOGON"
}
}
if [event\_id] in [4625, 4647] {
mutate {
add\_field =\> ["audit\_type", "FAILURE"]
add\_tag =\> "AUDIT\_FAILURE"
}
}
date {
match =\> ["created\_date", "M/d/yyyy H:mm:ss"]
}
}
}
But I have for the new rows only
\_grokparsefailure
What is the correct mode to use the date that I have in this format US 2/1/2019 4:38:41?
Thank you
Franco
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 2:31pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/4 "2019-04-30T14:31:47Z")
Are you getting \_grokparsefailure (which is what your wrote), or \_dateparsefailure (which is what you seem to be asking about).
---
### Author:  [@franco.federico](https://discuss.elastic.co/u/franco.federico)
#### Post date: [April 30, 2019, 2:52pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/5 "2019-04-30T14:52:56Z")
Thank you Badger. I just received \_grokparsefailure, so the match date is correct and why is it impossible to set the date received like timestamp of logstash?
Thank you
Franco
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 2:54pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/6 "2019-04-30T14:54:34Z")
If you are getting a grok parse failure then there will not be a created\_date field for the date filter to parse. You need to correct your pattern.
---
### Author:  [@franco.federico](https://discuss.elastic.co/u/franco.federico)
#### Post date: [April 30, 2019, 3:03pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/7 "2019-04-30T15:03:32Z")
So I correct the problem with this
```
date {
match => ["created_date", "M/d/yyyy H:mm:ss"]
target => "new_created_date"
}
```
Now I see the new\_created\_date, but the timestamp in which the data are loaded is equal to now instead of the correct data. Why?
Thank you
Franco
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 3:20pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/8 "2019-04-30T15:20:58Z")
The timestamp at which the date is loaded is determined by @timestamp. That is what the date filter writes to by default. If you give it another target then @timestamp will be the current time.
---
### Author:  [@franco.federico](https://discuss.elastic.co/u/franco.federico)
#### Post date: [April 30, 2019, 9:59pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/9 "2019-04-30T21:59:13Z")
Thank you Badger.
I found the solution to my problem.
If I set
```auto
date {
match => ["created_date", "M/d/yyyy H:mm:ss"]
}
```
I found in the today different \_grokparsefailure, but If I search in the time period of the creation date I found the data.
If I set instead
```auto
date {
match => ["created_date", "M/d/yyyy H:mm:ss"]
target => "new_created_date"
}
```
I found two rows with the same data but timestamp different
1 timestamp = the loading of the row with correct created\_date and new\_created data
1 timestamp = create\_date = new\_created\_date
So I prefer the first solution. Is it possible to delete \_grokparsefailure?
Thank you
Franco
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 10:24pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/10 "2019-04-30T22:24:52Z")
> [@franco.federico](#):
>
> Is it possible to delete \_grokparsefailure?
Yes, you can use
```
mutate { remove_tag => "_grokparsefailure" }
```
If that is the only tag on the event that leaves tags as an empty array. You can remove that using [this](https://discuss.elastic.co/t/how-to-overwrite-message-set-to-minus-if-it-is-not-overwrited-in-grok/138518/8).
I notice that in your original grok filter you tried to do remove\_tag =\> "\_grokparsefailure". That does not work the way you want. "Decoration" of events, which implements the common options like remove\_tag, only occurs if the filter succeeds. So it would only remove the \_grokparsefailure if it had not been added.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 28, 2019, 10:25pm UTC](https://discuss.elastic.co/t/losgstash-error-starting-with-expected-one-of-at-line-xx/179072/11 "2019-05-28T22:25:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.