Log4j logs and it import to logstash

Hi,

I had run a software which generates logs using log4j and saves them in a file ept.log the format is as follows

launch TIMESTAMP START: 2016.11.15 05:30:23
Level: Message
INFO: Batch lock retry wait = 30, max = 720
INFO: Launching Batch Validation
INFO: Timestamp: Nov 15,2016 05:30:24 AM
INFO: rules from properties null
INFO: ... Checking rules requested against rule info from the database
INFO: Starting run on oracle.jdbc.driver.OracleDriver jdbc:oracle:thin:@tch:1521:tch WEU_MAP_IW *******...
AUDIT: opening connection to user:CLONE_WSM url:jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=tch)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=tch)))
INFO: com.xyz.dataserver.jdbcserver.JdbcDatasetReader
INFO: create desc com.xyz.workspace.DBDescriptor: name=app, dataset=Url: jdbc:oracle:thin:@tch:1521:tch, driver: oracle.jdbc.driver.OracleDriver, User: WEU_MAP_IW, connection tag: dbConnectionString, converage=<CoverageDescriptor: extent=<com.xyz.dataserver.extent.RegionExtent region code:F1 region-to-admin query?:true region-to-carto query?:false loadBufferData?:true>, partialpolys=[], allCartoIds=[], onlyAdmins=false, includeChildrenInHierarchy=true, dbversion=0, countrystate=false, admincartoonly=false, longhaul=false, xzipcity=false, xpostal=false, crfs=[], queryMetadataOnly=false, (obs) loadAllMetadata=true, loadLocalMetadata=true, loadGlobalMetadata=true, loadTrafficData=true, loadTpegData=true, loadPhoneticData=false, loadGeoOverrideData=false, loadHistoryData=false, loadTopologySegmentData=false, queryAllLinks=true, queryLinkLimit=25000, deltas=false, unintDeltasOnly=false, unintDelta=false, unintDeltaPrev=false, synch=false, synchPrev=false, synchPrevPS=false, taskCode=0, scopeRectangles=null, syncType=null, versionToIgnoreForSync=0, allRoadNameIds=[], unrestrictedReadAllQuery=false, loadStatisticalAreas = false, loadBasicHeightData=false, filterLevel=NONE>
INFO: Creating Loader & Reader (bypassing app-server)
INFO: Loading an RMOB
INFO: creating the Reader
INFO: get info for Reader from properties
INFO: create a local Reader
INFO: set up the Reader by calling setFromReaderDescriptor()
INFO: Reader created
INFO: QUERY COUNT CartoFaces : 224815
INFO: 16950000 packets. Current packet is class com.xyz.kernel.flat.FlatPostalAreaAdminPlace [Loader] [tstamp: 3968 time: 3952.5 (1:05:52) split: 01.7] [Memory: 16.06G total: 22.20G free: 6.139G]
INFO: com.xyz.val.validation.BatchRoadPointValidation Validation statistics: 2 rule(s) applied, 0 exception(s) occurred (0 out of scope)
INFO: [BatchValidationCommand] [tstamp: 4992 time: 39.1 split: 00.0] [Memory: 18.95G total: 37.49G free: 18.54G]
INFO: Done executing all batch validations. [ValidationService] [tstamp: 4992 time: 38.7 split: 38.7] [Memory: 18.95G total: 37.49G free: 18.54G]
[save rule violations] [tstamp: 4998 time: 4997.6 (1:23:17) split: 4997.6 (1:23:17)] [Memory: 23.25G total: 37.49G free: 14.24G]
INFO:
INFO: Rule Code Rule Status
INFO: ---------- --------------------------------------------------------------------------- ------
INFO: PAD032 Point Address/Micro Point Address Consists of Illegal Characters OK
INFO: PRB046 Point Address Crossing Product Boundary OK
INFO: RDS069 Location Table Number Not Valid for Country or Admin OK
INFO:
INFO: SUMMARY REPORT
INFO:
INFO: BATCH VALIDATION CLASS NUM OF RULES NUM OF EXCEPTIONS
INFO: ------------------------------------------------------ ------------ ------------------
INFO: BatchCartoLinkValidation 1 0
INFO: BatchRoadPointValidation 2 0
INFO: TOTAL: 3 0
INFO: Batch Validation final status: SUCCESS
launch TIMESTAMP END: 2016.11.15 06:53:45

now how shall I extract data from it to visualize and update my config files and filters of logstash
so that if able to extract important metrics like Memory consumption at a particular timestamp or CPU usage or maybe you guys can help me figure out other aspects of it?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.