Multiline log parsing

Elastic Stack Logstash
1

Hi,
I am using grok pattern to get date from filename and that is working fine. Now i need to parse multiline log event along with date from filename.

Log:

8480 12:33:38 ERROR Error while rendering view: '/Views/Article/ads.cshtml' (model: 'Informa.Web.ViewModels.RelatedDealsModel, Informa.Web').

Exception: System.InvalidOperationException
Message: Error while rendering view: '/Views/Article/ArticleRelatedDeals.cshtml' (model: 'Informa.Web.ViewModels.ads.RelatedDealsModel, Informa.Web').

Source: Sitecore.Mvc
at Sitecore.Mvc.Presentation.ViewRenderer.Render(TextWriter writer)
at Informa.Library.CustomSitecore.Pipelines.RenderRenderings.ExecuteRenderer.Render(Renderer renderer, TextWriter writer, RenderRenderingArgs args) in D:\jenkins\jobs\prodFE\workspace\src\Informa.Library\CustomSitecore\Pipelines\RenderRenderings\ExecuteRenderer.cs:line 21

Nested Exception

Exception: System.NullReferenceException
Message: Object reference not set to an instance of an object.
Source: Informa.Library
at Informa.Library.Article.Companies.RelatedDealsService.b__3_2(Deal c) in D:\jenkins\jobs\prodFE\workspace\src\Informa.Library\Article\Companies\RelatedDealsService.cs:line 29
at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
at ASP._Page_Views_Article_ArticleRelatedDeals_cshtml.Execute() in D:\inetpub\wwwroot\INFORMA\Website\Views\Article\ArticleRelatedDeals.cshtml:line 8
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
at Sitecore.Mvc.Presentation.ViewRenderer.Render(TextWriter writer)

Please suggest grok to parse this multiline log.

2

I am using filebeat.yml as:
-
paths:
- /app/logs/sitecore/log.*
input_type: log
document_type: sitecore_log
pattern: '[[:digit:]]{4}[[:space:]][[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2}'
negate: true
match: next

logstash config:

 if [type] == "sitecore_log" {
    multiline {
      patterns_dir => "/etc/logstash/conf.d/patterns"
      pattern => "(^%{INT})|(^%{WORD})"
      negate => "true"
      what => "next"
    }
        grok {
         patterns_dir => ["/etc/logstash/conf.d/patterns"]
         match => [ "message", "(%{INT:pid}|%{WORD:heartbeat}) %{TIME:time} %{LOGLEVEL:loglevel} (%{GREEDY_DATA:sitecore_log}|%{GREEDY_DATA:rewrite_log})" ]
             }
        grok {
        match => ["source", "(%{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:day}\.%{HOUR:hour}%{MINUTE:minute}%{SECOND:second}\.txt$)"]
        add_field => ["timestamp", "%{year}-%{month}-%{day} %{time}"]
         }
        mutate {
        remove_field => ["year", "month", "day", "hour", "minute", "second"]
                }
        date {
        match => ["timestamp", "yyyy-MM-dd HH:mm:ss", "yyyy-MM-dd HH:mm:ss.SSSZ"]
        target => "@timestamp"
        }
        mutate {
        gsub => [
        "message","\n"," ",
        "message","\r",""
                ]
         }
   }
3

Thanks. This has been fixed. :slight_smile:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.