# Can only start a prospector when all related states are finished

**URL:** <https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 29, 2018, 8:48am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644 "2018-05-29T08:48:47Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![xdholy](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xdholy](https://discuss.elastic.co/u/xdholy)\
**Post date:** [May 29, 2018, 8:48am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/1 "2018-05-29T08:48:47Z")

</div>

Hello, I'm getting this error multiple times now.  
I realize I shouldn't define multiple prospectors over one file, but i put all my server's log into same log file.  
I need to use multiple prospector to define different log types.  
Is there anyway i can avoid this?  
Or can i start more than one filebeat program on the same server, using different configs?

Thanks.

Here's the prospector config.

- type: log  
enabled: true  
paths:

- type: log  
enabled: true  
paths:

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [June 1, 2018, 10:37am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/2 "2018-06-01T10:37:08Z")

</div>

Could you please provide more info on how do you want to filter messages from your log file?  
To me it seems like you have a log file which includes logs with different log levels, and you want to add fields to the messages depending on the log level. Is that correct?

If yes, you can achieve this by using one prospector and adding a processor which checks the log level and then applies the correct fields. See more on processors:

- [https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html](https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html)
- [https://www.elastic.co/guide/en/beats/filebeat/master/include-fields.html](https://www.elastic.co/guide/en/beats/filebeat/master/include-fields.html)

---

<div class="post-metadata">

**Author:** ![xdholy](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xdholy](https://discuss.elastic.co/u/xdholy)\
**Post date:** [June 4, 2018, 7:19am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/3 "2018-06-04T07:19:33Z")

</div>

Thank you.  
Yes, that's exactly what i meant.  
But I couldn't think of a way to get the different log levels from a log string before logstash grok.  
Therefore, i can't use the processor.  
That's why i use different prospector to add fields, and send to Logstash to apply different types of grok.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [June 4, 2018, 10:55am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/4 "2018-06-04T10:55:01Z")

</div>

Could you provide a few example logs? It would help me to come up with processor settings.

---

<div class="post-metadata">

**Author:** ![xdholy](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xdholy](https://discuss.elastic.co/u/xdholy)\
**Post date:** [June 4, 2018, 11:39am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/5 "2018-06-04T11:39:15Z")

</div>

OK.  
So it looks like this. 4 different log levels [DEBUG] [ERROR] [UNIQ] [TRACE] in the same file.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/5/c592ad1c4ab805660d91807186493247406c56a5.png)

Log Text:

2018-05-30 19:04:33.605 sceneserver[2102] ERROR: [LUA][Random] RandomSucc player:20000032 XD level:17 group:1 zone:20 VIP:10 source:23 type:67  
2018-06-01 16:02:42.646 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000032 XD level:17 group:1 zone:20 VIP:8 source:6 gear:4999 diamond:9040  
2018-06-02 12:02:42.646 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000021 HOLLY level:17 group:1 zone:20 VIP:8 source:6 gear:4999 diamond:10040  
2018-06-02 19:03:06.691 sceneserver[2102] DEBUG: [LUA][Charge] PlayerCharge player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:7 gear:9999 diamond:21040  
2018-06-04 20:06:32.956 sceneserver[2102] UNIQ: [LUA][LEVEL] LEVELUP player:20000021 HOLLY level:17 group:1 zone:20 VIP:10 source:35 score:220 area:4  
2018-06-03 19:04:33.605 sceneserver[2102] DEBUG: [LUA][Random] RandomSucc player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:23 type:67  
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!  
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!  
2018-06-04 19:04:33.605 sceneserver[2102] ERROR: [LUA][Active] Player Active Config Error!  
2018-06-04 19:04:33.605 sceneserver[2102] TRACE: Player 20000001 login  
2018-06-04 20:06:32.956 sceneserver[2102] DEBUG: [LUA][Game] Fish player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:35 AddExp:400  
2018-06-04 20:06:32.956 sceneserver[2102] UNIQ: [LUA][LEVEL] LEVELUP player:20000006 Korry level:17 group:1 zone:20 VIP:10 source:35 score:0 area:3

usual struct is:  
timestamp-server-id-loglevel-prog-module-...

Now, i can only get the log level after send it to logstash, after grok.

---

<div class="post-metadata">

**Author:** ![xdholy](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xdholy](https://discuss.elastic.co/u/xdholy)\
**Post date:** [June 4, 2018, 11:39am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/6 "2018-06-04T11:39:56Z")

</div>

Thanks for your time. The log file looks like the pic above.

---

<div class="post-metadata">

**Author:** ![xdholy](https://avatars.discourse-cdn.com/v4/letter/x/7cd45c/32.png) [@xdholy](https://discuss.elastic.co/u/xdholy)\
**Post date:** [June 13, 2018, 8:18am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/7 "2018-06-13T08:18:46Z")

</div>

Could you help me with this? Thanks a lot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 11, 2018, 8:18am UTC](https://discuss.elastic.co/t/can-only-start-a-prospector-when-all-related-states-are-finished/133644/8 "2018-07-11T08:18:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
