Logstash config file execution error?

Elastic Stack Logstash
1

Hi,
I have one config file which is used to analyse the slowlogs and keep it in file.The code for that is

input {
file {
path => "C:\Users\571952\Downloads\elasticsearch-5.1.1\elasticsearch-5.1.1\logs\elasticsearch_index_search_slowlog"
start_position => "beginning"
}
}

filter {
grok { # parses the common bits
match => [ "message", "[%{URIHOST}:%{ISO8601_SECOND}][%{LOGLEVEL:log_level}]
[%{DATA:es_slowquery_type}]\s*[%{DATA:es_host}]\s*[%{DATA:es_index}]\s*[%{DATA:es_shard}]\stook[%{DATA:es_duration}],\stook_millis[%{DATA:es_duration_ms:float}],\stypes[%{DATA:es_types}],\sstats[%{DATA:es_stats}],\ssearch_type[%{DATA:es_search_type}],\stotal_shards[%{DATA:es_total_shards:float}],\ssource[%{GREEDYDATA:es_source}],\sextra_source[%{GREEDYDATA:es_extra_source}]"]
}

mutate {
gsub => [
"source_body", "], extra_source[$", ""
]
}
}

output {
file {
path => "C:\Users\571952\Desktop\logstash-5.1.1\just_queries"
codec => "json_lines"

}
}

when i tried executing it in my cmd prompt it is showing error like this

[2017-01-04T18:30:32,032][ERROR][logstash.agent ] Pipeline aborted due to error
{:exception=>#<RegexpError: premature end of char-class: /], extra_source[$/>, :backtrac
e=>["org/jruby/RubyRegexp.java:1424:in initialize'", "C:/Users/571952/Desktop/logstash-5 .1.1/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/logstash/filters/mutat e.rb:196:inregister'", "org/jruby/RubyArray.java:1653:in each_slice'", "C:/Users/57195 2/Desktop/logstash-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/lo gstash/filters/mutate.rb:184:inregister'", "C:/Users/571952/Desktop/logstash-5.1.1/logs
tash-core/lib/logstash/pipeline.rb:230:in start_workers'", "org/jruby/RubyArray.java:161 3:ineach'", "C:/Users/571952/Desktop/logstash-5.1.1/logstash-core/lib/logstash/pipeline
.rb:230:in start_workers'", "C:/Users/571952/Desktop/logstash-5.1.1/logstash-core/lib/lo gstash/pipeline.rb:183:inrun'", "C:/Users/571952/Desktop/logstash-5.1.1/logstash-core/l
ib/logstash/agent.rb:292:in `start_pipeline'"]}
[2017-01-04T18:30:32,141][INFO ][logstash.agent ] Successfully started Logstash
API endpoint {:port=>9600}
[2017-01-04T18:30:35,036][WARN ][logstash.agent ] stopping pipeline {:id=>"main
"}

My slowlog content is like this:

[2016-12-28T15:53:21,341][DEBUG][index.search.slowlog.query] [vVhZxH7] [sw][0] took[184.7micros], took_millis[0], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{
"ext" : { }
}],

2

Since you didn't post your configuration or the logs as preformatted text some parts might've disappeared, but I suspect the problem is that you need to escape the square brackets in the gsub regexp. Square brackets have a special meaning in regexps and if you want literal matches you need to escape the brackets.

3

Hi,
Sry.. i cant able to get your point like "escape the square brackets in gsub regexp". Can you explain in brief?
Thanks

4

Change

mutate {
  gsub => [
    "source_body", "], extra_source[$", ""
  ]
}

to

mutate {
  gsub => [
    "source_body", "\], extra_source\[$", ""
  ]
}

Perhaps the dollar sign should be escaped too? I don't know what your data looks like so I can't tell.

5

Thanks,that change which you had mentioned is worked fine. But i cant able to see just_queries in my output folder.It seems my logstash is started successfully but nothing happened ..

Pasted image754×265 25.1 KB

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.