# Panic error

**URL:** <https://discuss.elastic.co/t/panic-error/81600>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 7, 2017, 12:00pm UTC](https://discuss.elastic.co/t/panic-error/81600 "2017-04-07T12:00:27Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![AndreAga](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@AndreAga](https://discuss.elastic.co/u/AndreAga)\
**Post date:** [April 7, 2017, 12:00pm UTC](https://discuss.elastic.co/t/panic-error/81600/1 "2017-04-07T12:00:27Z")

</div>

Hi,  
I receive the following panic error:

```
2017/04/07 11:35:58.534945 outputs.go:108: INFO Activated elasticsearch as output plugin.
2017/04/07 11:35:58.535113 publish.go:295: INFO Publisher name: weblogic
2017/04/07 11:35:58.535350 async.go:63: INFO Flush Interval set to: 1s
2017/04/07 11:35:58.535364 async.go:64: INFO Max Bulk Size set to: 50
2017/04/07 11:35:58.535904 beat.go:221: INFO filebeat start running.
2017/04/07 11:35:58.536076 registrar.go:85: INFO Registry file set to: /home/.../filebeat/data/registry
2017/04/07 11:35:58.536371 registrar.go:106: INFO Loading registrar data from /home/.../filebeat/data/registry
2017/04/07 11:35:58.536719 registrar.go:123: INFO States Loaded from registrar: 5
2017/04/07 11:35:58.536745 crawler.go:38: INFO Loading Prospectors: 3
2017/04/07 11:35:58.536853 prospector_log.go:61: INFO Prospector with previous states loaded: 1
2017/04/07 11:35:58.537001 prospector.go:124: INFO Starting prospector of type: log; id: 13661519049829733422
2017/04/07 11:35:58.537094 prospector_log.go:61: INFO Prospector with previous states loaded: 4
2017/04/07 11:35:58.537221 spooler.go:101: INFO Stopping spooler
2017/04/07 11:35:58.537585 prospector.go:232: INFO Stopping Prospector: 13661519049829733422
2017/04/07 11:35:58.537692 registrar.go:236: INFO Starting Registrar
2017/04/07 11:35:58.537725 sync.go:41: INFO Start sending events to output
2017/04/07 11:35:58.537756 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017/04/07 11:35:58.538572 prospector.go:134: INFO Prospector channel stopped
panic: send on closed channel

goroutine 104 [running]:
panic(0x9345e0, 0xc4202a7c10)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/elastic/beats/filebeat/beater.(*spoolerOutlet).OnEvent(0xc42018f480, 0xc4202977a0, 0x7f6d498d24b0)
    /go/src/github.com/elastic/beats/filebeat/beater/channels.go:57 +0x194
github.com/elastic/beats/filebeat/prospector.(*Prospector).updateState(0xc4201e1e00, 0xc4202977a0, 0x27, 0x61394)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:203 +0x6e
github.com/elastic/beats/filebeat/prospector.(*Prospector).startHarvester(0xc4201e1e00, 0xc4200d79b0, 0x27, 0x61394, 0x0, 0xc831e0, 0xc420213a00, 0x1524, 0x1f, 0xed079709e, ...)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:303 +0x17d
github.com/elastic/beats/filebeat/prospector.(*ProspectorLog).harvestExistingFile(0xc4201d3980, 0xc4200d79b0, 0x27, 0x0, 0x0, 0xc831e0, 0xc420213a00, 0x1524, 0x1f, 0xed079709e, ...)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector_log.go:265 +0xb9b
github.com/elastic/beats/filebeat/prospector.(*ProspectorLog).scan(0xc4201d3980)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector_log.go:247 +0x717
github.com/elastic/beats/filebeat/prospector.(*ProspectorLog).Run(0xc4201d3980)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector_log.go:81 +0xce
github.com/elastic/beats/filebeat/prospector.(*Prospector).Run(0xc4201e1e00)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:170 +0x4f
github.com/elastic/beats/filebeat/prospector.(*Prospector).Start.func2(0xc4201e1e00)
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:161 +0x51
created by github.com/elastic/beats/filebeat/prospector.(*Prospector).Start
    /go/src/github.com/elastic/beats/filebeat/prospector/prospector.go:162 +0x1a1

```

This is my config file:

```
filebeat.prospectors:

- input_type: log
  paths:
    - /.../weblogic.log
  fields:
    source_type: weblogic
  multiline.pattern: '^#'
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 100000

- input_type: log
  paths:
    - /.../*/*.log
 exclude_files: ['isp_.*']
  fields:
    source_type: logback-application
  multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2} '
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 100000
  harvester_limit: 5
  close_eof: true

- input_type: log
  paths:
    - /.../*/isp_*.log
  fields:
    source_type: logback-isp-application
  multiline.pattern: '^[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3} '
  multiline.negate: true
  multiline.match: after
  multiline.max_lines: 100000
  harvester_limit: 1
  close_eof: true

output.elasticsearch:
  hosts: ["http://H1:9200", "http://H2:9200", "http://H3:9200", "http://H4:9200"]
  indices:
    - index: "weblogic-%{+yyyy.MM.dd}"
      when.contains:
        fields.source_type: "weblogic"
    - index: "logback-application-%{+yyyy.MM.dd}"
      when.contains:
        fields.source_type: "logback-application"
    - index: "logback-application-%{+yyyy.MM.dd}"
      when.contains:
        fields.source_type: "logback-performance"
    - index: "logback-application-%{+yyyy.MM.dd}"
      when.contains:
        fields.source_type: "logback-isp-application"
  pipelines:
    - pipeline: "weblogic"
      when.equals:
        fields.source_type: "weblogic"
    - pipeline: "logback-application"
      when.equals:
        fields.source_type: "logback-application"
    - pipeline: "logback-performance"
      when.equals:
        fields.source_type: "logback-performance"
    - pipeline: "logback-isp-application"
      when.equals:
        fields.source_type: "logback-isp-application"
  template.name: "filebeat"
  template.path: "filebeat.template.json"
  template.overwrite: true
  loadbalance: true
  worker: 2

```

And this is OS Version:  
Red Hat Enterprise Linux Server release 5.8 (Tikanga)

I'm using Filebeat 5.3.

I noticed that if I comment the second prospector all works fine, but I don't know why.

Thanks

---

<div class="post-metadata">

**Author:** ![AndreAga](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@AndreAga](https://discuss.elastic.co/u/AndreAga)\
**Post date:** [April 7, 2017, 12:45pm UTC](https://discuss.elastic.co/t/panic-error/81600/2 "2017-04-07T12:45:18Z")

</div>

I tried the same configuration with Filebeat 5.2.2 and it works.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 10, 2017, 7:49am UTC](https://discuss.elastic.co/t/panic-error/81600/3 "2017-04-10T07:49:46Z")

</div>

Sounds to be the same panic as here [https://github.com/elastic/beats/issues/3917](https://github.com/elastic/beats/issues/3917) But so far we assumed it only happens with empty paths. I will look into this.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 10, 2017, 7:57am UTC](https://discuss.elastic.co/t/panic-error/81600/4 "2017-04-10T07:57:33Z")

</div>

@AndreAga There is an indentation issue in prospector 2. I assume it should not be related to this, but could you try to fix it?

---

<div class="post-metadata">

**Author:** ![AndreAga](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@AndreAga](https://discuss.elastic.co/u/AndreAga)\
**Post date:** [April 10, 2017, 8:13am UTC](https://discuss.elastic.co/t/panic-error/81600/5 "2017-04-10T08:13:46Z")

</div>

Hi ruflin, I confirm that in the second prospector the '/\*/' points to some empty dirs.  
The indentation error is related to the copy/paste. In the original file the indentation is ok 🙂

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 10, 2017, 8:28am UTC](https://discuss.elastic.co/t/panic-error/81600/6 "2017-04-10T08:28:17Z")

</div>

Good to know. I need to find out why a prospector stops when a path is empty.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 18, 2017, 1:39pm UTC](https://discuss.elastic.co/t/panic-error/81600/7 "2017-04-18T13:39:26Z")

</div>

@AndreAga I just opened [https://github.com/elastic/beats/pull/4037](https://github.com/elastic/beats/pull/4037) to fix the issue. But I'm not 100% sure yet this will solve your issue. The issue that is solved in this PR is that in case one prospector has an invalid configuration and filebeat is stopped, it makes sure the shutdown happens without a panic. You mentioned that in your case it works with 5.2.2 but not 5.3. This would indicate that your prospector config is probably fine and it is a different issue.

Could you try the following for me? Remove all prospector configurations except the one that you removed before and see if filebeat starts properly? And if not, what error you get?

---

<div class="post-metadata">

**Author:** ![AndreAga](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@AndreAga](https://discuss.elastic.co/u/AndreAga)\
**Post date:** [April 18, 2017, 2:03pm UTC](https://discuss.elastic.co/t/panic-error/81600/8 "2017-04-18T14:03:16Z")

</div>

Thank you, ruflin. I'll let you know as soon as I'll be able to test it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 16, 2017, 2:14pm UTC](https://discuss.elastic.co/t/panic-error/81600/9 "2017-05-16T14:14:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
