I was testing multiline in file section but DID NOT have a filter section. I included an empty filter section and its working.
I am trying to get LogStash to combine a log file with multilines defining an event. The "trigger" for a new event has the term "ICSubCall" embedded as a word in the log line.
Here is my config input segment:
input {
file {
codec => plain {
charset => "CP1252"
}
path => "/mnt/hgfs/VMShared/logs/uvcs/uvcs*"
ignore_older => 5184000
close_older => 15
type => "uvcs"
start_position => "beginning"
codec => multiline {
pattern => "/\bICSubCall\b/"
negate => true
what => previous
}
sincedb_path => "/root/since_db_files/sincedb_uvcs"
}
}
And here is a segment of the logs:
Wed Jun 29 01:32:59; errno=0: ICSubCall args=7,name=UO.SPIDER,
Wed Jun 29 01:32:59; errno=0: arg[0]=HTTP_USER_AGENTþJNI_QUERYþSPIDER_VERSIONþRGWHOSTþRGWADDRþPATH_INFO,
Wed Jun 29 01:32:59; errno=0: arg[1]=redbeans=1þ1þ101þdsb-p07.maeagle.corpþ192.168.18.11þ/dmc/PRODUCTS:ProductDetail,this.getCalcProdInfo,
Wed Jun 29 01:32:59; errno=0: arg[2]=,
Wed Jun 29 01:32:59; errno=0: arg[3]=,
Wed Jun 29 01:32:59; errno=0: arg[4]=,
Wed Jun 29 01:32:59; errno=0: arg[5]=reqParentIDþorigProdIdsþprodSiteTypeþprocessNameþisMAProdþseqIDþprodLineTypeþprodOverridesþlangCodeþprodIdsþprodTaxPctþspecProcFormulaIDþprodRetailsþchargeNoCostFlagþpayingIDþmarkUpPctþpromoCodeþenableFreeShippingþlastTimeUpdatedþdistIDþmpaIDþsiteTypeþorderSourceþstdCashBackAmtþdiscPctþprodQtysþprodCostsþempIDþshipStateþshipCountryþprodTaxCostþorigQtysþtlsFitFlagþprodBvsþcampaignRestrictþpersonalConsþpcIdþsiteCntryþprodTaxAmtþmerchCountryþlastDateUpdatedþprodIbvsþmaImageURLþisCampaignþHID_FORM_INSTþredbeansþcomo,
Wed Jun 29 01:32:59; errno=0: arg[6]=dd0ac865-4760-4d4f-8f3c-3f010859241bþC13029ýC6217þUýUþþ1ý1þþþ0ý0þENGþC13029ýC6217þþ0ý0þþ0þ648034486þþþ0þ01:29amý01:28amþ648034486þþUþþ0ý0þþ7ý2þþþBCþCANþþ7ý2þþþ0ý0þ0þþUSAþþCANþ06/29/2016ý06/29/2016þþ//images.marketamerica.com/images/USA/shopping/products/þ0ý0þþ1þ1,
Wed Jun 29 01:32:59; errno=0: uvapi_server non execute command.
Wed Jun 29 01:32:59; errno=0: api_slave: function code=52.
Wed Jun 29 01:32:59; errno=0: ICSubCall args=7,name=UO.SPIDER,
Wed Jun 29 01:32:59; errno=0: arg[0]=HTTP_USER_AGENTþJNI_QUERYþSPIDER_VERSIONþRGWHOSTþRGWADDRþPATH_INFO,
Wed Jun 29 01:32:59; errno=0: arg[1]=redbeans=1þ1þ101þdsb-p09.maeagle.corpþ192.168.18.13þ/dmc/SERVICES:WebServices,this.getCreditUser,
Wed Jun 29 01:32:59; errno=0: arg[2]=,
Wed Jun 29 01:32:59; errno=0: arg[3]=,
Wed Jun 29 01:32:59; errno=0: arg[4]=,
Wed Jun 29 01:32:59; errno=0: arg[5]=langCodeþpcIDþisSiteBrandedþsiteTypeþreqParentIDþsiteCountryþcreditUserþrepIDþHID_FORM_INSTþredbeansþcomo,
Wed Jun 29 01:32:59; errno=0: arg[6]=CHIþ1897767þþUþ0dd585ab-a999-4575-91a8-4a60bf14b085þTWNþþ3431475þþ1þ1,
Wed Jun 29 01:32:59; errno=0: uvapi_server non execute command.
Wed Jun 29 01:32:59; errno=0: api_slave: function code=52.
As you can see, I'm trying to use the regex word boundary pattern as the "new" event trigger. I have tried changing negate to true/false and what to next/previous; but LogStash will not combine the lines into a single message.
I have checked the regex against the on-line test utility ( https://regex101.com/ ) and it works as a match there.
It seems fairly straight forward, but I just can't seem to get it to work. Any help would be most appreciated.
Thanks,
Dave