# Logstash filter for Jboss Fuse 6.2.1

**URL:** <https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116>\
**Category:** Logstash\
**Created:** [March 15, 2018, 2:45pm UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116 "2018-03-15T14:45:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![solo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solo/32/29084_2.png) [@solo](https://discuss.elastic.co/u/solo)\
**Post date:** [March 15, 2018, 2:45pm UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/1 "2018-03-15T14:45:09Z")

</div>

I've tried to create a filter for Logstash that can consume my Jboss Fuse log file.

My filter is working, and I get data into Kibana. But I was wondering if is possible to split the message into:  
-Date  
-Message Type  
-Log Subsystem  
-Route  
-Java class  
-Message Text

Example from fuse.log:

> 2018-03-15 06:04:18,795 | INFO | ../../ftp/input | route105 ID-domain.com-227-10-27620 | com.domain.JavaClassImpl | Some message from Java

> 2018-03-15 15:27:35,464 | WARN | ActiveMQ Task-1 | | org.apache.activemq.transport.failover.FailoverTransport | Failed to connect to [tcp://localhost:61616] after: 10 attempt(s) continuing to retry.

My filter:

```
input {
        beats {
                port => "5044"
        }
}
filter {

   grok {
     match =>
        {
          "message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:level}\s+\[%{DATA:className}\]%{SPACE}%{GREEDYDATA:message}"
        }
   }

}
output {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
                document_type => "%{[metadata][type]}"
        }
}

```

What I see in Kibana:

```
host:
    selite
tags:
    beats_input_codec_plain_applied, _grokparsefailure
@timestamp:
    March 15th 2018, 15:27:36.381
source:
    /usr/lib/jboss-fuse/data/log/fuse.log
@version:
    1
message:
    2018-03-15 15:27:35,464 | WARN | ActiveMQ Task-1 | | org.apache.activemq.transport.failover.FailoverTransport | Failed to connect to [tcp://localhost:61616] after: 10 attempt(s) continuing to retry.
offset:
    76,910
beat.hostname:
    selite
beat.name:
    selite
beat.version:
    6.2.2
prospector.type:
    log
_id:
    hEgPKmIBNzHwb5UAITuV
_type:
    %{[metadata][type]}
_index:
    filebeat-2018.03.15
_score:
    -

```

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 15, 2018, 7:44pm UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/2 "2018-03-15T19:44:51Z")

</div>

> [@solo](#):
>
> \_grokparsefailure

the `_grokparsefailure` tag indicates that your grok pattern failed to parse the input; you can use the [Grok Constructor](http://grokconstructor.appspot.com/do/match) to test an expression against a number of log lines.

* * *

Since your data is very well-formatted and consistently delimited, you may have better luck with [`logstash-filter-dissect`](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html), which uses delimiter-splitting under the hood and is much easier to understand because we don't have to worry so much about exact format of each field:

```auto
# ...
filter {
  dissect {
    mapping => {
      "message" => "%{timestamp} | %{level} | %{subsystem} | %{route} | %{class} | %{message}"
    }
  }
  date {
    match => ["timestamp", "ISO8601"]
  }
}

```

_EDIT: original was missing `=>` in date filter_

---

<div class="post-metadata">

**Author:** ![solo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solo/32/29084_2.png) [@solo](https://discuss.elastic.co/u/solo)\
**Post date:** [March 16, 2018, 9:38am UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/3 "2018-03-16T09:38:44Z")

</div>

Thank you yaauie.

Updated filter:  
GNU nano 2.8.6 File: logstash-syslog.conf

```
input {
        beats {
                port => "5044"
        }
}
filter {
  dissect {
    mapping => {
      "message" => "%{timestamp} | %{level} | %{subsystem} | %{route} | %{class} | %{message}"
    }
  }
  date {
    match ["timestamp", "ISO8601"]
  }
}
output {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
                document_type => "%{[metadata][type]}"
        }
}

```

I tried to use that filter but I got error when starting up:

```
sindre@selite:~$ tail -f /var/log/logstash/logstash-plain.log
[2018-03-16T10:34:52,532][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-16T10:34:52,665][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.2.2"}
[2018-03-16T10:34:52,821][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-16T10:34:52,857][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 13, column 11 (byte 205) after filter {\n dissect {\n mapping => {\n \"message\" => \"%{timestamp} | %{level} | %{subsystem} | %{route} | %{class} | %{message}\"\n }\n }\n date {\n match ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
[2018-03-16T10:35:22,329][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-03-16T10:35:22,334][INFO][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-03-16T10:35:22,580][WARN][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-03-16T10:35:22,790][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"6.2.2"}
[2018-03-16T10:35:22,964][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-03-16T10:35:23,034][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 13, column 11 (byte 205) after filter {\n dissect {\n mapping => {\n \"message\" => \"%{timestamp} | %{level} | %{subsystem} | %{route} | %{class} | %{message}\"\n }\n }\n date {\n match ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
```

---

<div class="post-metadata">

**Author:** ![solo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solo/32/29084_2.png) [@solo](https://discuss.elastic.co/u/solo)\
**Post date:** [March 16, 2018, 10:14am UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/4 "2018-03-16T10:14:15Z")

</div>

I found out that the filter was missing =\> at date match.  
Now it works!

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 16, 2018, 3:06pm UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/5 "2018-03-16T15:06:39Z")

</div>

Derp. I've updated my previous comment to be correct.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 13, 2018, 3:06pm UTC](https://discuss.elastic.co/t/logstash-filter-for-jboss-fuse-6-2-1/124116/6 "2018-04-13T15:06:59Z")

</div>

Closed on April 13, 2018, 3:06pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.