Logstash applying grok to only specific log lines

Elastic Stack Logstash
1

Hi,

why logstash applying grok for only only few log lines even though I have grok patterns for all log lines and I can run through same logs using grok debugger and its working fine, I think its behaving like unpredictable way or am I missing anyth

Summary

This text will be hidden

ing?

this is my logstash debug log:

[2019-05-17T11:54:12,123][DEBUG][logstash.pipeline        ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:12.013Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 14 03:36:15
Summary

[spoiler]dllabsw1 account-server: 172.24.7.109

[/spoiler] - - [14/May/2019:03:36:15 +0000] "PUT /d3/26974/.expiring_objects/1557791933" 201 - "-" "-" "container-updater 13974" 0.0007 "-" 35828 0", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}
[2019-05-17T11:54:12,124][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:11.934Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 7 03:15:09

Summary

mstpol13 object-server: 10.188.42.67

    • [07/May/2019:10:15:09 +0000] "DELETE /d345/11314/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" 404 70 "DELETE http://localhost/v1/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" "txb694b5d6133045268cffc-005cd15aad" "proxy-server 20232" 0.0235 "-" 32623 0", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}
      [2019-05-17T11:54:12,127][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:12.009Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 14 03:36:05
Summary

dllabsw1 account-server: 172.24.7.109

    • [14/May/2019:03:36:05 +0000] "HEAD /d3/33163/ACC_VZ_8" 204 - "HEAD http://localhost/v1/ACC_VZ_8?format=json" "tx4dce18b99d8d40b582337-005cda37a5" "proxy-server 13994" 0.0018 "-" 35858 -", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}
      [2019-05-17T11:54:12,127][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:12.014Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 14 03:36:16
Summary

dllabsw1 account-server: 172.24.7.109

    • [14/May/2019:03:36:16 +0000] "GET /d3/49561/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?marker=&prefix=&end_marker=&format=json" "txcf347ccd4b5247f1975e7-005cda37b0" "proxy-server 13978" 0.0004 "-" 35821 -", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}
      [2019-05-17T11:54:12,127][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:12.016Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 14 03:36:14
Summary

dllabsw1 object-server: 172.24.7.108

Summary

dllabsw1 container-server: 172.24.7.109

    • [14/May/2019:03:36:16 +0000] "GET /d3/38894/.misplaced_objects/1557802800" 404 - "GET http://localhost/v1/.misplaced_objects/1557802800?marker=&states=listing&prefix=&end_marker=&format=json" "tx916fef5c496149da8790d-005cda37b0" "proxy-server 13978" 0.0006 "-" 36135 0", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}
      [2019-05-17T11:54:12,128][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2019-05-17T18:54:12.020Z, "@version"=>"1", "tags"=>["swift_paco_logs"], "message"=>"May 13 03:30:30
Summary

dllabsw1 proxy-server: 172.24.7.97 172.24.7.97

13/May/2019/03/30/30 PUT /v1/ACC_VZ_1/CONT_VZ_2/file/NzYwMDA2NzI2OHwwMDAwMDAwLTAwMDAtMDE3Ny0wMDA3LTYyOTc1fDIwMTktMDUtMTRUMDM6MzA6Mjla/0 HTTP/1.0 201 - - - 5243780 - - txe8bfda41221243dc8fa93-005cd8e4d5 - 0.1125 - x-delete-at:1557804629 1557718229.895745039 1557718230.008220911 0", "host"=>"0.0.0.0", "path"=>"/opt/swiftlogs/test.log"}}

Thanks
Chandra

Summary

This text will be hidden

2

output received:
[2019-05-17T11:54:12,270][DEBUG][logstash.pipeline ] output received {"event"=>{"logsource"=>"mstpol13", "host"=>"0.0.0.0", "Object"=>"http://localhost/v1/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40", "response_time"=>0.0235, "txn"=>"txb694b5d6133045268cffc-005cd15aad", "@version"=>"1", "method"=>"DELETE", "tags"=>["swift_paco_logs", "swift_object_parsed"], "response"=>"404", "@timestamp"=>2019-05-17T18:54:11.934Z, "message"=>["May 7 03:15:09 mstpol13 object-server: 10.188.42.67 - - [07/May/2019:10:15:09 +0000] "DELETE /d345/11314/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" 404 70 "DELETE http://localhost/v1/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" "txb694b5d6133045268cffc-005cd15aad" "proxy-server 20232" 0.0235 "-" 32623 0", "07/May/2019:10:15:09 +0000", "/d345/11314/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40", "70", "DELETE", ""proxy-server 20232"", ""-" 32623 0"], "path"=>"/opt/swiftlogs/test.log", "program"=>["object-server", "object-server"], "logdate"=>2019-05-07T10:15:09.000Z}}
[2019-05-17T11:54:12,275][DEBUG][logstash.pipeline ] output received {"event"=>{"logsource"=>"dllabsw1", "host"=>"0.0.0.0", "Object"=>"http://localhost/v1/ACC_VZ_8?format=json", "response_time"=>0.0018, "txn"=>"tx4dce18b99d8d40b582337-005cda37a5", "@version"=>"1", "method"=>"HEAD", "tags"=>["swift_paco_logs", "swift_account_parsed"], "response"=>"204", "@timestamp"=>2019-05-17T18:54:12.009Z, "message"=>["May 14 03:36:05 dllabsw1 account-server: 172.24.7.109 - - [14/May/2019:03:36:05 +0000] "HEAD /d3/33163/ACC_VZ_8" 204 - "HEAD http://localhost/v1/ACC_VZ_8?format=json" "tx4dce18b99d8d40b582337-005cda37a5" "proxy-server 13994" 0.0018 "-" 35858 -", "14/May/2019:03:36:05 +0000", "/d3/33163/ACC_VZ_8", "HEAD", ""proxy-server 13994"", ""-" 35858 -"], "path"=>"/opt/swiftlogs/test.log", "program"=>["account-server", "account-server"], "logdate"=>2019-05-14T10:36:05.000Z}}
[2019-05-17T11:54:12,276][DEBUG][logstash.pipeline ] output received {"event"=>{"logsource"=>"dllabsw1", "host"=>"0.0.0.0", "Object"=>"http://localhost/v1/.misplaced_objects?marker=&prefix=&end_marker=&format=json", "response_time"=>0.0004, "txn"=>"txcf347ccd4b5247f1975e7-005cda37b0", "@version"=>"1", "method"=>"GET", "tags"=>["swift_paco_logs", "swift_account_parsed"], "response"=>"404", "@timestamp"=>2019-05-17T18:54:12.014Z, "message"=>["May 14 03:36:16 dllabsw1 account-server: 172.24.7.109 - - [14/May/2019:03:36:16 +0000] "GET /d3/49561/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?marker=&prefix=&end_marker=&format=json" "txcf347ccd4b5247f1975e7-005cda37b0" "proxy-server 13978" 0.0004 "-" 35821 -", "14/May/2019:03:36:16 +0000", "/d3/49561/.misplaced_objects", "GET", ""proxy-server 13978"", ""-" 35821 -"], "path"=>"/opt/swiftlogs/test.log", "program"=>["account-server", "account-server"], "logdate"=>2019-05-14T10:36:16.000Z}}
[2019-05-17T11:54:12,276][DEBUG][logstash.pipeline ] output received {"event"=>{"logsource"=>"dllabsw1", "host"=>"0.0.0.0", "Object"=>"http://localhost/v1/.misplaced_objects/1557802800?marker=&states=listing&prefix=&end_marker=&format=json", "response_time"=>0.0006, "txn"=>"tx916fef5c496149da8790d-005cda37b0", "@version"=>"1", "method"=>"GET", "tags"=>["swift_paco_logs", "swift_container_parsed"], "response"=>"404", "@timestamp"=>2019-05-17T18:54:12.018Z, "message"=>["May 14 03:36:16 dllabsw1 container-server: 172.24.7.109 - - [14/May/2019:03:36:16 +0000] "GET /d3/38894/.misplaced_objects/1557802800" 404 - "GET http://localhost/v1/.misplaced_objects/1557802800?marker=&states=listing&prefix=&end_marker=&format=json" "tx916fef5c496149da8790d-005cda37b0" "proxy-server 13978" 0.0006 "-" 36135 0", "14/May/2019:03:36:16 +0000", "/d3/38894/.misplaced_objects/1557802800", "GET", ""proxy-server 13978"", ""-" 36135 0"], "path"=>"/opt/swiftlogs/test.log", "program"=>["container-server", "container-server"], "logdate"=>2019-05-14T10:36:16.000Z}}

3

What does your configuration, especially the grok filter, look like?

4

@Badger,

here is the grok patterns

SWIFT_PROXY_HEAD %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: - - %{NOTSPACE:message} %{WORD:method} %{UNIXPATH:Object} %{WORD:message}/%{NUMBER:httpversion} %{NUMBER:response} - %{WORD:message} - - - - %{NOTSPACE:txn} - %{NUMBER:response_time} %{GREEDYDATA:message}

SWIFT_PROXY_GET_POST %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{HOSTNAME} %{HOSTNAME} %{NOTSPACE:message} %{WORD:method} %{UNIXPATH:Object} %{WORD:message}/%{NUMBER:httpversion} %{NUMBER:response} - - - - %{NUMBER:message} - %{NOTSPACE:txn} - %{NUMBER:response_time} %{GREEDYDATA:message}


SWIFT_PROXY_PUT %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{HOSTNAME} %{HOSTNAME} %{NOTSPACE:message} %{WORD:method} %{UNIXPATH:Object} %{WORD:message}/%{NUMBER:httpversion} %{NUMBER:response} - - - %{NUMBER:message} - - %{NOTSPACE:txn} - %{NUMBER:response_time} %{GREEDYDATA:message}

SWIFT_PROXY_DELETE %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{HOSTNAME} %{HOSTNAME} %{NOTSPACE:message} %{WORD:method} %{UNIXPATH:Object} %{WORD:message}/%{NUMBER:httpversion} %{NUMBER:response} - - - - - - %{NOTSPACE:txn} - %{NUMBER:response_time} - - %{GREEDYDATA:message}



#######################
#OBJECT-SERVER: PROGRAM
#######################

SWIFT_OBJECT_GET %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:message}" %{NUMBER:response} %{NUMBER:message} "%{WORD:message} %{NOTSPACE:Object}" "%{NOTSPACE:txn}" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}


SWIFT_OBJECT_PUT_DEL %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:message}" %{NUMBER:response} (-|%{NUMBER}) "%{WORD:message} %{NOTSPACE:Object}" "%{NOTSPACE:txn}" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}
SWIFT_OBJECT_POST %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:message}" %{NUMBER:response} %{NUMBER:message} "%{WORD:message} %{NOTSPACE:Object}" "%{NOTSPACE:txn}" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}

##########################
#CONTAINER-SERVER: PROGRAM
##########################

SWIFT_CONTAINER_HEAD_GET_PUT_DELETE %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:message}" %{NUMBER:response} - "%{WORD:message} %{NOTSPACE:Object}" "%{NOTSPACE:txn}" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}


#########################
#ACCOUNT-SERVER: PROGRAM
#########################

SWIFT_ACCOUNT_HEAD_GET_PUT_DELETE_0 %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:message}" %{NUMBER:response} - "%{WORD:message} %{NOTSPACE:Object}" "%{NOTSPACE:txn}" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}
SWIFT_ACCOUNT_HEAD_GET_PUT_DELETE_1 %{SYSLOGTIMESTAMP:logdate} %{SYSLOGHOST:logsource} %{SYSLOGPROG:program}: %{IPV4} - - \[%{HAPROXYDATE:message}\] "%{WORD:method} %{UNIXPATH:Object}" %{NUMBER:response} - "-" "-" %{QS:message} %{NUMBER:response_time} %{GREEDYDATA:message}

SWIFT_AP_REST_ALL %{SYSLOGTIMESTAMP:logdate} %{GREEDYDATA:msg}
5

these are my sample log lines:
May 7 03:15:09 mstpol13 object-server: 10.188.42.67 - - [07/May/2019:10:15:09 +0000] "DELETE /d345/11314/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" 404 70 "DELETE http://localhost/v1/ACC_439/be495727-fcc8-4f2b-a3f8-e1305175b5bc/1554307448596%3A27a29814-dd8f-43ee-b768-19af98bf1d07%3A40" "txb694b5d6133045268cffc-005cd15aad" "proxy-server 20232" 0.0235 "-" 32623 0
May 14 03:36:05 dllabsw1 account-server: 172.24.7.109 - - [14/May/2019:03:36:05 +0000] "HEAD /d3/33163/ACC_VZ_8" 204 - "HEAD http://localhost/v1/ACC_VZ_8?format=json" "tx4dce18b99d8d40b582337-005cda37a5" "proxy-server 13994" 0.0018 "-" 35858 -
May 14 03:36:15 dllabsw1 account-server: 172.24.7.109 - - [14/May/2019:03:36:15 +0000] "PUT /d3/26974/.expiring_objects/1557791933" 201 - "-" "-" "container-updater 13974" 0.0007 "-" 35828 0
May 14 03:36:16 dllabsw1 account-server: 172.24.7.109 - - [14/May/2019:03:36:16 +0000] "GET /d3/49561/.misplaced_objects" 404 - "GET http://localhost/v1/.misplaced_objects?marker=&prefix=&end_marker=&format=json" "txcf347ccd4b5247f1975e7-005cda37b0" "proxy-server 13978" 0.0004 "-" 35821 -
May 14 03:36:14 dllabsw1 object-server: 172.24.7.108 - - [14/May/2019:03:36:14 +0000] "DELETE /d10/60128/ACC_VZ_2/CONT_VZ_5/10a29c70-6590-11e9-93da-632735cf0214/1557632120301%3ARGVmYXVsdC8zMzAwMDY5MDU4%3Af7b7b9d1-7466-11e9-bed6-679edf32393d" 204 - "DELETE http://localhost/v1/ACC_VZ_2/CONT_VZ_5/10a29c70-6590-11e9-93da-632735cf0214/1557632120301%3ARGVmYXVsdC8zMzAwMDY5MDU4%3Af7b7b9d1-7466-11e9-bed6-679edf32393d" "tx65fde37e031b428dadf00-005cda37ae" "proxy-server 35676" 0.0270 "-" 36158 0
May 14 03:36:16 dllabsw1 container-server: 172.24.7.109 - - [14/May/2019:03:36:16 +0000] "GET /d3/38894/.misplaced_objects/1557802800" 404 - "GET http://localhost/v1/.misplaced_objects/1557802800?marker=&states=listing&prefix=&end_marker=&format=json" "tx916fef5c496149da8790d-005cda37b0" "proxy-server 13978" 0.0006 "-" 36135 0
May 13 03:30:30 dllabsw1 proxy-server: 172.24.7.97 172.24.7.97 13/May/2019/03/30/30 PUT /v1/ACC_VZ_1/CONT_VZ_2/file/NzYwMDA2NzI2OHwwMDAwMDAwLTAwMDAtMDE3Ny0wMDA3LTYyOTc1fDIwMTktMDUtMTRUMDM6MzA6Mjla/0 HTTP/1.0 201 - - - 5243780 - - txe8bfda41221243dc8fa93-005cd8e4d5 - 0.1125 - x-delete-at:1557804629 1557718229.895745039 1557718230.008220911 0

6

Please let me know if you want my config file.

when I was testing above log lines I noticed inconsistent outputs, some times it parsing 2 log lines and some times 3/4 log lines.

Thanks
Chandra

7

I want to see the grok filter at least.

8

conf file:

input {
file {
    path => "/opt/swiftlogs/test.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    #sincedb_path  => "/opt/data/logstash/plugins/inputs/file/proxy-aco-logs.log"
    tags => ["swift_paco_logs"]
}
}

#input {
 # beats {
  #  port => 5044
   # }
#}


filter {
   if "swift_paco_logs" in [tags] and  "proxy-server:" in [message] {
            grok {
                patterns_dir => ["/opt/logstash/patterns"]
                break_on_match => true
                match => {
                      "message" => ["%{SWIFT_PROXY_HEAD}",
                        "%{SWIFT_PROXY_GET_POST}",
                        "%{SWIFT_PROXY_PUT}",
                        "%{SWIFT_PROXY_DELETE}"
                           ]
                     }
                add_tag => [ "swift_proxy_parsed" ]
            }
           }
  if "swift_paco_logs" in [tags] and "object-server:" in [message] {
        grok {
                patterns_dir => ["/opt/logstash/patterns"]
                match => {
                        "message" => ["%{SWIFT_OBJECT_GET}",
                                "%{SWIFT_OBJECT_PUT_DEL}",
                                "%{SWIFT_OBJECT_POST}"
                                ]
                }
                add_tag => [ "swift_object_parsed" ]
        }
 }

  if "swift_paco_logs" in [tags] and "container-server:" in [message] {
        grok {
                patterns_dir => ["/opt/logstash/patterns"]
                match => {
                        "message" => [ "%{SWIFT_CONTAINER_HEAD_GET_PUT_DELETE}" ]
                }
                add_tag => [ "swift_container_parsed" ]
        }
 }

  if "swift_paco_logs" in [tags] and "account-server:" in [message] {
        grok {
                patterns_dir => ["/opt/logstash/patterns"]
                match => {
                        "message" => ["%{SWIFT_ACCOUNT_HEAD_GET_PUT_DELETE_0}",
                                     "%{SWIFT_ACCOUNT_HEAD_GET_PUT_DELETE_1}"
                                      ]
                }
                add_tag => [ "swift_account_parsed" ]
        }
 }
  if "swift_proxy_parsed" not in [tags]  and "swift_object_parsed" not in [tags]  and "swift_container_parsed" not in [tags]  and "swift_account_parsed" not in [tags] {
      grok {
          patterns_dir => ["/opt/logstash/patterns"]
                match => {
                      "message" => [ "%{SWIFT_AP_REST_ALL}" ]
                      }
                add_tag => [ "swift_rest_all" ]
         }
 }
date {
match => ["logdate", "MMM dd HH:mm:ss", "MMM  d HH:mm:ss"]
target => "logdate"
timezone => "PST8PDT"
}
#date {
#match => ["logdate", "MMM  d HH:mm:ss"]
#target => "logdate"
#timezone => "PST8PDT"
#}


 mutate {
       convert => {
                "httpversion" => "float"
                "status_code" => "integer"
                "response_time" => "float"
                  }
      remove_field => [ "httpversion","haproxy_hour", "haproxy_milliseconds", "haproxy_minute", "haproxy_month", "haproxy_monthday", "haproxy_second", "haproxy_time", "haproxy_year" ]
        }
}


output {
 #if "swift_paco_logs" in [tags] {
   elasticsearch {
      hosts => [ "http://IP:9200" ]
      index => "swift_paco"
      #index => "swift_ap_log_write_alias"
      template => "/opt/logstash/config/templates/swift_ap_log_sizing.json"
      template_name => "swift_ap_log_template"
      template_overwrite => true
      #user => '***'
      #password => '***'
      #ssl => true
      #cacert => "/opt/logstash-6.1.3/config/ssl/logstash.pem"
      #ssl_certificate_verification => false
  }
stdout {}
 #}
}
9

using pipeline as below.

  • pipeline.id: swiftaplogs
    pipeline.workers: 2
    path.config: "/opt/logstash/config/all_config/swift_aco_proxy_log.conf"
10

I am amazed it parses anything at all! I let it run for several minutes and then killed it without anything being parsed. Firstly, you should read Do you grok Grok? on the elastic blog. Seriously, go and read that. My post will still be here when you get back.

Next, anchor all of your patterns with ^, to match start of line. Your patterns match the entire line, so tell grok that. Change

SWIFT_PROXY_DELETE %{SYSLOGTIMESTAMP:logdate} [...]

to

SWIFT_PROXY_DELETE ^%{SYSLOGTIMESTAMP:logdate} [...]

At that point I still have logstash hung, burning 100% of the CPU and get no events. The next problem ... do not use UNIXPATH! It is insanely expensive when it does not match.

I replaced UNIXPATH with NOTSPACE and logstash processed the 7 test lines in your first post in a fraction of a second. Now UNIX paths can have spaces in them, so the two are not equivalent, but the point is use a different regexp that matches your data, do not use UNIXPATH.

1 Like
11

I really appreciate your comments @Badger.. I will read and change as you suggested and update you.

but one qq.. Did you notices same behavior what I have seen before changing ^ and UNIXPATH ?

Thanks
Chandra

12

I am on a single CPU machine, if it is stuck back-tracking on one pattern, no other patterns can be processed. If you had a multi-CPU machine then it would be possible for events to be processed on one CPU which another CPU were stuck back-tracking on PROXY patterns. It would depend on the order of lines and how quickly they failed to match the PROXY patterns. I would not be at all surprised for it to be data dependent.

PS. You have posted a commented-out password in your config. You might want to change that to

  password => "*"

or something like that.

13

Thanks @Badger for your help.. after applying you suggested changes its working perfect.

Thanks
Chandra

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.