Before I go down the path of creating something custom to bring in W3C format access logs generated from .NET https server apps on Linux, I wanted to make sure there wasn’t some easier way to do it with an existing integration.
The logs are the default setting with X-Forwarded-For, so they would be similar to what is collected from IIS, though they have a much different naming.
Any insight before I venture down the custom log path is greatly appreciated.
Here is the header at beginning of log file showing fields, and a few creatively redacted lines (this is a dev box so the x-forwarded-for is just “-” but it contains an IP on prod boxes and is often populated from cf-connecting-ip rather than x-forward-for):
So the closest Integration we have is the IIS integration, it matches a number of the most popular pattern
It supports a number of patterns but unfortunately it does not appear to support your customer pattern.
I would suggest
A) Loading the integration and cloning the Ingest Pipeline and add a GROK pattern that matches your specific pattern.
or
B) Rearrange your output to match one of the OOTB patterns
C) Though not recommended you can edit the ingest pipeline from the integration but those changes will be overwritten if / when you upgrade the integration.
I can make the log match the pattern. After that would it be as simple as adding the Linux paths and patterns for the logs to the IIS Access Logs paths?
So matching your Application Logs to one of the Existing Supporting Patterns, then yes you can then use the IIS Integration and point it to the logs paths...
If you provide a few examples after you re-format I can show you how to do a quick test.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.