I have this log
[2018-12-07 00:00:06,048][xx.xxx.xxx.xxxxxxxxxxxxxxxxx.xxxx.xxxxl ][INFO ][http-apr-8080-exec-583 ] - TDR (id= '4048814086', channel= '', msisdn= 'xxxxxxxxxx', service= 'nombreservicio', transactionDate= 'Fri Dec 07 00:00:06 ART 2018', duration= '140', parameters= ',xxxxxxxxxx', response= '<?xml version="1.0" encoding="UTF-8" standalone="yes"?> VEQ_Producto CONTATs VEQ_Nivel EN VEQ_NroCliente 091574128 VEQ_TipoDoc CU VEQ_NroDoc 30708714409 VEQ_Grupo E VEQ_Nro_Cuenta 97289305 VEQ_Fidel_Oferta ', errorCode= '', errorDescription= '', instance= 'nombreinstancia')
I’m tying using grok dubbuger. In grok debbuger work fine
%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD} - TDR%{SPACE}(id= '%{NUMBER:id}',%{SPACE}channel= '%{DATA:channel},%{SPACE}msisdn= '%{DATA:msisdn}',%{SPACE}service= '%{DATA:service}',%{SPACE}transactionDate= '%{DATA:transactionDate}',%{SPACE}duration= '%{NUMBER:duration}',%{SPACE}parameters= '%{DATA:parameters},%{SPACE}response= '%{DATA:response},%{SPACE}errorCode= '%{DATA:errorCode},%{SPACE}errorDescription= %{DATA:errorDescription},%{SPACE}instance= '%{DATA:instance}'
Ythis is the result
{
"SYSLOG5424SD": [
[
"[2018-12-07 00:00:06,048]",
"[xx.xxx.xxx.xxxxxxxxxxxxxxxxx.xxxx.xxxxl ]",
"[INFO ]",
"[http-apr-8080-exec-583 ]"
]
],
"DATA": [
[
"2018-12-07 00:00:06,048",
"xx.xxx.xxx.xxxxxxxxxxxxxxxxx.xxxx.xxxxl ",
"INFO ",
"http-apr-8080-exec-583 "
]
],
"SPACE": [
[
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" ",
" "
]
],
"id": [
[
"4048814086"
]
],
"BASE10NUM": [
[
"4048814086",
"140"
]
],
"channel": [
[
"'"
]
],
"msisdn": [
[
"xxxxxxxxxx"
]
],
"service": [
[
"nombreservicio"
]
],
"transactionDate": [
[
"Fri Dec 07 00:00:06 ART 2018"
]
],
"duration": [
[
"140"
]
],
"parameters": [
[
",xxxxxxxxxx'"
]
],
"response": [
[
"<?xml version="1.0" encoding="UTF-8" standalone="yes"?> VEQ_Producto CONTATs VEQ_Nivel EN VEQ_NroCliente 091574128 VEQ_TipoDoc CU VEQ_NroDoc 30708714409 VEQ_Grupo E VEQ_Nro_Cuenta 97289305 VEQ_Fidel_Oferta '"
]
],
"errorCode": [
[
"'"
]
],
"errorDescription": [
[
"''"
]
],
"instance": [
[
"nombreinstancia"
]
]
}
But in logstash doesn’t work
Logstash.conf
grok { match => { "message" => "%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD} - TDR%{SPACE}(id= '%{NUMBER:id}',%{SPACE}channel= '%{DATA:channel},%{SPACE}msisdn= '%{DATA:msisdn}',%{SPACE}service= '%{DATA:service}',%{SPACE}transactionDate= '%{DATA:transactionDate}',%{SPACE}duration= '%{NUMBER:duration}',%{SPACE}parameters= '%{DATA:parameters},%{SPACE}response= '%{DATA:response},%{SPACE}errorCode= '%{DATA:errorCode},%{SPACE}errorDescription= %{DATA:errorDescription},%{SPACE}instance= '%{DATA:instance}')" }}
first the error was
[2018-12-07T09:24:47,277][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x71a40865 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id="06db14c6405fcd300c69958d36c303ab30fbda5b2e4a04b6d10bd77a6998796e", @klass=LogStash::Filters::Grok, @metric_events=#LogStash::Instrument::NamespacedMetric:0x10a254ec, @filter=<LogStash::Filters::Grok match=>{"message"=>"%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD}%{SYSLOG5424SD} - TDR%{SPACE}\\(id= '%{NUMBER:id}',%{SPACE}channel= '%{DATA:channel},%{SPACE}msisdn= '%{DATA:msisdn}',%{SPACE}service= '%{DATA:service}',%{SPACE}transactionDate= '%{DATA:transactionDate}',%{SPACE}duration= '%{NUMBER:duration}',%{SPACE}parameters= '%{DATA:parameters},%{SPACE}response= '%{DATA:response},%{SPACE}errorCode= '%{DATA:errorCode},%{SPACE}errorDescription= %{DATA:errorDescription},%{SPACE}instance= '%{DATA:instance}')"}, id=>"06db14c6405fcd300c69958d36c303ab30fbda5b2e4a04b6d10bd77a6998796e", enable_metric=>true, periodic_flush=>false, patterns_files_glob=>"", break_on_match=>true, named_captures_only=>true, keep_empty_captures=>false, tag_on_failure=>["_grokparsefailure"], timeout_millis=>30000, tag_on_timeout=>"_groktimeout">>", :error=>"unmatched close parenthesis: /(?:\[(?:.?)\]+)(?:\[(?:.?)\]+)(?:\[(?:.?)\]+)(?:\[(?:.?)\]+) - TDR(?:\s)\(id= '(?NUMBER:id(?:(?:(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+))))))',(?:\s*)channel= '(?<DATA:channel>.?),(?:\s)msisdn= '(?<DATA:msisdn>.?)',(?:\s)service= '(?<DATA:service>.?)',(?:\s)transactionDate= '(?<DATA:transactionDate>.?)',(?:\s)duration= '(?NUMBER:duration(?:(?:(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+))))))',(?:\s*)parameters= '(?<DATA:parameters>.?),(?:\s)response= '(?<DATA:response>.?),(?:\s)errorCode= '(?<DATA:errorCode>.?),(?:\s)errorDescription= (?<DATA:errorDescription>.?),(?:\s)instance= '(?<DATA:instance>.*?)')/m", :thread=>"#<Thread:0x5ac15438 run>"}
Then, now, i don’t see error, but, the message is not working
