Can only start a prospector when all related states are finished

Elastic Stack Beats
1

Hello, I'm getting this error multiple times now.
I realize I shouldn't define multiple prospectors over one file, but i put all my server's log into same log file.
I need to use multiple prospector to define different log types.
Is there anyway i can avoid this?
Or can i start more than one filebeat program on the same server, using different configs?

Thanks.

Here's the prospector config.

  • type: log
    enabled: true
    paths:

    • /log/server*.log.*
      fields:
      level: log

  • type: log
    enabled: true
    paths:

    • /log/server*.log.*
      fields:
      level: error
2

Could you please provide more info on how do you want to filter messages from your log file?
To me it seems like you have a log file which includes logs with different log levels, and you want to add fields to the messages depending on the log level. Is that correct?

If yes, you can achieve this by using one prospector and adding a processor which checks the log level and then applies the correct fields. See more on processors:

3

Thank you.
Yes, that's exactly what i meant.
But I couldn't think of a way to get the different log levels from a log string before logstash grok.
Therefore, i can't use the processor.
That's why i use different prospector to add fields, and send to Logstash to apply different types of grok.

4

Could you provide a few example logs? It would help me to come up with processor settings.

5

OK.
So it looks like this. 4 different log levels [DEBUG] [ERROR] [UNIQ] [TRACE] in the same file.

image
image.png1647×263 30.7 KB

Log Text:

2018-05-30 19:04:33.605 sceneserver[2102] ERROR: [LUA][Random] RandomSucc player:20000032 XD level:17 group:1 zone:20 VIP:10 source:23 type:67
2018-06-01 16:02:42.646 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000032 XD level:17 group:1 zone:20 VIP:8 source:6 gear:4999 diamond:9040
2018-06-02 12:02:42.646 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000021 HOLLY level:17 group:1 zone:20 VIP:8 source:6 gear:4999 diamond:10040
2018-06-02 19:03:06.691 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:7 gear:9999 diamond:21040
2018-06-04 20:06:32.956 sceneserver[2102] UNIQ: [LUA][LEVEL] LEVELUP player:20000021 HOLLY level:17 group:1 zone:20 VIP:10 source:35 score:220 area:4
2018-06-03 19:04:33.605 sceneserver[2102] DEBUG: [LUA][Random] RandomSucc player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:23 type:67
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!
2018-06-04 19:04:33.605 sceneserver[2102] TRACE: Player 20000001 login
2018-06-04 20:06:32.956 sceneserver[2102] DEBUG: [LUA][Game] Fish player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:35 AddExp:400
2018-06-04 20:06:32.956 sceneserver[2102] UNIQ: [LUA][LEVEL] LEVELUP player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:35 score:0 area:3

usual struct is:
timestamp-server-id-loglevel-prog-module-...

Now, i can only get the log level after send it to logstash, after grok.

6

Thanks for your time. The log file looks like the pic above.

7

Could you help me with this? Thanks a lot.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.