How can I increase the size of bytes that a log can have?

Elastic Stack Beats
1

I need help, I am receiving a log of 1300 lines, I am receiving a parse error, for this I did tests and these logs were significantly reduced and with that the error disappears since what I am removing does not affect the structure of the log. For which I only have one theory left... and that is that the size of my log is exceeding the limit. Use the max_bytes: 10485760 like this:

filebeat.inputs:

- type: filestream
  id: error-app1
  enabled: true
  paths:
    - C:\Users\60085367\Desktop\ServerErrorParsi\pruebaslogs\*
  
  max_bytes: 10485760

  processors:
  - add_fields: 
      fields: 
        V_APP_NAME: error.Log.Tester1

  parsers:
    - multiline:
        type: pattern
        pattern: "^<log4j:event.*" 
        negate: true
        match: after
2

Not sure what data do you have and which error has been shown. Just take the longest line and save in a file.
You have this param: multiline.max_lines: 500

Formore info check documentation.

3

I had already observed that information but I couldn't draw a conclusion since I see that they use the type syslog, tcp input, udp input. Confirm me if I'm right, for the filestream type I can only apply the max_line inside the multiline parsers:.

I understand that for the filestream it is done like this:

  parsers:
    - multiline:
        type: pattern
        pattern: "^<log4j:event.*" 
        negate: true
        match: after
        max_line: 1500
4

Yes, you are using filestream. Set little more than you expect

   - multiline:
        max_line: 2500
        max_bytes: 504857600

Set debug mode and will see is it applied.

1 Like
5

How to apply debug mode? I still don't know much about it.
So far I have only used the command filebeat.exe -c filebeat.yml

6

This will display activities on the screen
filebeat.exe -c filebeat.yml -e if you don't see then add -d "*"
or use log

logging.level: debug
logging.to_files: true
logging.files:
  path: /path/dir
  name: filebeat.log
  keepfiles: 7
  permissions: 0644
7

I got this result, but I don't see the change, the error still persists. My error is the parsers of the xml, but this is due to the fact that it does not reach the closing of the event and the message, firstly because it exceeds 500 lines.

image
image1351×200 17.7 KB

8

Data path locked by another beat. The most likely you have left the FB service active.

9

Sorry, I already found the error. I was using max_line and it was max_lines....

10

Now is everything OK?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.