Logstash filter for django logs not working


(Luvpreet Singh) #1
filter {
   if[type] == "django" {

        grok {

          patterns_dir => ["/opt/logstash/patterns/"]
          match => [ "message" , "%{LOGLEVEL:loglevel} %{DJANGOTIMESTAMP:timestamp},{INT:pid} %{WORD:origin} %{INT:id} %{INT:number}" ]
        }
}
}

This is the filter I am using for my django logs, and below is an entry of django logs,

INFO 2017-05-16 16:47:16,087 views 64864 140257592080128 https://play.google.com/store/apps/details?id=com.VoDrive&referrer=referral_code%3DrVEj

But it is not working,

      {
    "_index" : "django_indexer",
    "_type" : "django",
    "_id" : "AVwacAEnGhPsaxxZT_9J",
    "_score" : 1.0,
    "_source" : {
      "@timestamp" : "2017-05-18T07:22:41.927Z",
      "offset" : 144,
      "@version" : "1",
      "input_type" : "log",
      "beat" : {
        "hostname" : "DHARI-Inspiron-3542",
        "name" : "DHARI-Inspiron-3542",
        "version" : "5.4.0"
      },
      "host" : "DHARI-Inspiron-3542",
      "source" : "/var/log/django2.log",
      "message" : "INFO 2017-05-16 07:10:45,178 views 37497 140647884060416 https://play.google.com/store/apps/details?id=com.VoDrive&referrer=referral_code%3DP5E",
      "type" : "django",
      "tags" : [
        "beats_input_codec_plain_applied",
        "_grokparsefailure"
      ]
    }
  },

Here is the pattern,

DJANGOTIMESTAMP %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}
DJANGOLOG %{LOGLEVEL:loglevel} %{DJANGOTIMESTAMP:timestamp},{INT:pid} %{WORD:origin} %{INT:id} %{INT:number}

It says the grok parser has failed. What I am doing wrong here, and what do I need to do more in the gork parser ?


(Mark Walkom) #2

Please don't post multiple threads on the same problem - Grok parse failure for django logs


(Mark Walkom) #3