Filebeat : How to get multiline logs together (with pattern) in one block

Elastic Stack Beats
1

Hi everyone, im working on extracting log by filebeat and processing it by ELK stack.
every thing works exept that the logs are readed line per line and not by block with the specified pattern . thank you in advance.
this is my filebeat.inputs :

  • type: log
    enabled: true
    paths:
    • C:\Users\ismailm\Desktop\lineup*.log
      multiline:
      pattern: 'http>^['
      negate: true
      match: after
      flush_pattern: 'http> Response'

log example :
http> [02/03/2022 08:47:28] GET /star/multisession/x-707780/CommandProcessor?RGICommand=inquiryNodoRvFiltro&RGIResponseOk=${EXTRES}/portafoglio/common/ToolFinderRV.html&LABEL_NODO_ID=PrpvIdpv&LABEL_NODO_DESCR=PrpvCdescrizione1&showOnlyValidNodes=true&LABEL_NOME_FORM=RGI_FORM_DT
RGI> (0 rows in 1~1 msec) Select * From PRMANDATO Where ( IDPVFISICO=? And dInizio<=? And (dFine>? Or dFine Is Null) ) Order By IDCOMPAGNIA [1,{d '2022-03-02'},{d '2022-03-02'}]
http> Response in 15 msec. RGIResponseOk="/star/res2/portafoglio/common/ToolFinderRV.html"

2

I think this pattern is wrong, the ^ should be used when your pattern needs to start with some expression, also if I'm not wrong you need to escape the square bracket.

Can you change it to ^http>\s\[ and try again?

1 Like
3

it works for me and gives me the same result.
the problem that when the logs generated line per line and the filebeat send hits for every new line but i need to send a hit for every multiline block with the specified pattern

4

What is the result you are getting?

If you are getting an output line by line from filebeat, then the multiline pattern is not working.

What is the filebeat version you are using?

5

version : 8.0.1
the output is a message contain a block as expected when i enter the log(manually) by block and get all the block in one hit , otherwise in real logs the lines will be added line per line in the log file , and the file beat collect them one by one , or i need the filebeat to collect the block when the pattern is completed , because i m applyin a grok pattern on it

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.