Send whole log file to logstash


(Stefano) #1

Hi all, I'm quite new to Filebeat, and i need some help.
I'm trying to send log to Logstash from Filebeat, and receive the message by mail.
I need to send the entire LOG (custom) file in the message, if it contain the word "errors".

If I use:
include_lines: ['errors']
logstash send me only the row containing "errors".

If I use
multiline.pattern: '.errors.'
multiline.negate: false
multiline.match: after
logstash send me a email for each rows of the log file.

This is an example of one of the file i need to parse, someone can help me?

***************************LogStart*******************************  

START IMPORT : Tue 01/29/2019 1:11:11.66


Application Server LOG directory: c:\PerformServer\import-export\log\
DB Server: "SRV\QA"
DB Name: "Planning"

Call ARCA

0 rows updated on table [VASA_Import_Sell Out Qty FP] for changed value
232404 rows inserted into table [VASA_Import_Sell Out Qty FP]

Call ASA

Retail Console Utility
Version: 8.2 CU2
Build Number: 8.2.6589.02620

Server: http://localhost:5500/WebServices/OlapService.svc/OlapEndPoint
Model: MASTER

*** Command result:
Application: Planning

*** Actions ***
*** End actions ***
Import command completed with warnings.
Writeback operations
Data import results
Details:

Entity 'Sell Out Qty FP': 11806 errors.
Entity 'Sell Out Qty FP' imported.

Process of the cube for site 'Planning' failed.
Process not executed.
The site has been processed with errors.
Multi-Processing of 'Planning_Olap' failed
An error on the Olap server was encountered. Planning_Olap: An error on the Olap server was encountered. Errors in the OLAP storage engine: The attribute key cannot be found when processing: Table: 'BR2_Perform_x0020_Measure_x0020_Group_x0020_4', Column: 'D_SKU_MemberId', Value: '492682'. The attribute is 'SKU'.
Errors in the OLAP storage engine: The process operation ended because the number of errors encountered during processing reached the defined limit of allowable errors for the operation.
Errors in the OLAP storage engine: An error occurred while processing the 'Sell Out Qty FP' partition of the 'Perform Measure Group 4' measure group for the 'BR2' cube from the Planning_BR2_Olap database.
Internal error: The operation terminated unsuccessfully.
Server: The current operation was cancelled because another operation in the transaction failed.
Errors in the OLAP storage engine: An error occurred while processing the 'Sell Out Qty FP' partition of the 'Perform Measure Group 4' measure group for the 'BR4' cube from the Planning_Olap database.
Errors in the OLAP storage engine: An error occurred while processing the 'Sell Out Qty FP' partition of the 'Perform Measure Group 4' measure group for the 'AdminCube' cube from the Planning_Olap database.
Errors in the OLAP storage engine: An error occurred while processing the 'Sell Out Qty FP' partition of the 'Perform Measure Group 4' measure group for the 'BR1' cube from the Planning_Olap database.
Errors in the OLAP storage engine: An error occurred while processing the 'Sell Out Qty FP' partition of the 'Perform Measure Group 4' measure group for the 'BR3' cube from the Planning_Olap database.

Cube process failed.

*** Command result end.

Call ARCA Feedback

LogStart****
END IMPORT : Tue 01/29/2019 1:11:51.56



(Pier-Hugues Pellerin) #2

If I understand correctly, correct me if I am wrong, you want to send the whole logfile only if there is an error in the logs?

Since Filebeat and Logstash both operate on log line I don't think its possible with your current log format.


(Stefano) #3

Thanks, you saved my time.
I solved sending only the row with the "error" and the name of the log file that say me where is the problem.


(system) closed #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.