Hi all,
I'm completely new to Grok and have been tasked with writing a seemingly simple grok filter to identify whether our servers are in sync or not. In short, every day each server runs a database sync check and pumps out a txt file with the results. A sync fail file looks like this, note the "Databases appear out of sync".
##############################################################################
# Executing dbSyncCheck Maintenance Task
# Sat Mar 10 03:33:01 2018
Report:
synchcheck_basic.pl started Sat Mar 10 03:33:02 EST 2018
Performing basic comparison between datastores AppServer on server.name (local) and AppServer on server.name.2
*** Table appears out of sync: BWAS.ACCESS_DEVICE
*** Table appears out of sync: BWAS.ALT_ADDR
*** Table appears out of sync: BWAS.ALT_VM_MAILSERVER
*** Table appears out of sync: BWAS.AS_USER
*** Table appears out of sync: BWAS.AUTH_LOCKOUT_COUNT
*** Table appears out of sync: BWAS.CALL_LOG
*** Table appears out of sync: BWAS.DEF_VM_MAILSERVER
*** Table appears out of sync: BWAS.DEVICE_FILE_MAP
*** Table appears out of sync: BWAS.DEVICE_FILE_MAP_DEVICE
*** Table appears out of sync: BWAS.ENDPOINT
*** Table appears out of sync: BWAS.ENDPOINT_ADDR
*** Table appears out of sync: BWAS.EN_TRANSDATA
*** Table appears out of sync: BWAS.EXTENSION
*** Table appears out of sync: BWAS.LOGIN
*** Table appears out of sync: BWAS.MAIL_ADDR
*** Table appears out of sync: BWAS.MESSAGE_SUMMARY
*** Table appears out of sync: BWAS.REG_ADDR
*** Table appears out of sync: BWAS.SH_INTF
*** Table appears out of sync: BWAS.SPEED_CODE
*** Table appears out of sync: BWAS.USER_ASSN_FTR_SVC
*** Table appears out of sync: BWAS.USER_AUTH_FTR
*** Table appears out of sync: BWAS.USER_CALLP_POLICY
*** Table appears out of sync: BWAS.USER_CALL_POLICIES
*** Table appears out of sync: BWAS.USER_DEVICE_POLICIES
*** Table appears out of sync: BWAS.USER_FTR
*** Table appears out of sync: BWAS.USER_FTR_ATTR
*** Table appears out of sync: BWAS.USER_SVC
*** Table appears out of sync: BWAS.USER_SVC_ATTR
*** Table appears out of sync: BWAS.USER_VP_GREETING_FILE_MAP
*** Table appears out of sync: BWAS.VM
*** Table appears out of sync: BWAS.VM_GREETING
*** Activity on table prevents comparison: BWAS.DMS_COMPLETED_EVENT
*** Activity on table prevents comparison: BWAS.DMS_EVENT
*** WARNING: Databases appear out of sync
Recommended Actions:
Resynchronize the databases (with importdb.pl)
Error:
# Maintenance Task dbSyncCheck execution FAILED with error code 6
# Sending bwMaintenanceTaskFailure trap
##############################################################################
------------- A sync success looks like this: -------------
##############################################################################
# Executing dbSyncCheck Maintenance Task
# Sun Apr 1 03:33:02 2018
Report:
synchcheck_basic.pl started Sun Apr 1 03:33:02 BST 2018
Performing basic comparison between datastores AppServer on server.name (local) and AppServer on server.name.2
*** Activity on table prevents comparison: BWAS.DMS_COMPLETED_EVENT
Databases appear in sync, but there is activity on some tables that prevent their comparison
Recommended Actions:
Rerun synchcheck_basic.pl in period of low usage
Error:
# Maintenance Task dbSyncCheck execution FAILED with error code 1
# Sending bwMaintenanceTaskFailure trap
##############################################################################
I'm able to pull the time and date, the year and the hostname of the server using:
%{SYSLOGTIMESTAMP:datestamp}\s*%{GREEDYDATA}\s*%{YEAR:year}\s*%{GREEDYDATA}AppServer on\s*%{HOSTNAME:server}\s*
I also want to pull the "Databases appear out of sync" and "Databases appear in sync" from each file using a single filter if at all possible, but when I configure a custom pattern, Grok scans the next line for matches and then gives up without checking the rest of the file. What would you suggest and please advise of any better ways to write the above filter!
Thanks,
Steve