# Logstash filter verifier - any working example? **URL:** \ **Category:** Logstash\ **Created:** [February 16, 2016, 12:09pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880 "2016-02-16T12:09:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 16, 2016, 12:09pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/1 "2016-02-16T12:09:27Z")
Continuing the discussion from [Logstash test with rspec](https://discuss.elastic.co/t/logstash-test-with-rspec/41478/2): The tool [https://github.com/magnusbaeck/logstash-filter-verifier](https://github.com/magnusbaeck/logstash-filter-verifier) seems to be exactly what I was looking for: - easy install (in compare to rspec) - based on logstash library - simple definition of expected output to provided input But unfortunately I am unable to run any test with success and always get this error: ```auto 2016/02/16 12:58:29 Waiting for child with pid 58724 to terminate. Error running Logstash: exit status 1. The process gave no output. ``` Do I need to provide the input somehow next to the argument? Or am I overlooking something? sample example I am trying to run: ```auto logstash-filter-verifier test.json logstash.conf ``` test.json ```auto { "codec": "json", "ignore": ["host", "@timestamp"], "input": [ "{\"message\": \"This is a test message\"}" ], "expected": [ { "message": "This is a test message" } ] } ``` logstash.conf ```auto input { stdin { codec => "json" } } filter { } output { stdout { codec => "json" } } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 12:34pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/2 "2016-02-16T12:34:55Z")
Please use Logstash Filter Verifier 1.0.1 which gives better error output when Logstash fails. It also fixes an incompatibility issue with Logstash 2.2.0 (which has also been fixed in Logstash 2.2.1). Secondly, don't include any inputs or outputs in the configuration file(s) you feed LFV with. Just filters. The documentation should be more clear about this. ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 16, 2016, 12:41pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/3 "2016-02-16T12:41:27Z")
Hi Magnus, ok, removed the input and output from my logstash.conf, I am on 2.2.1, but still the same ```auto lukas@apulanta ~/P/k/l/logstash-filter-verifier_1.0.1_linux_amd64> ./logstash-filter-verifier --version logstash-filter-verifier 1.0.1 lukas@apulanta ~/P/k/l/logstash-filter-verifier_1.0.1_linux_amd64> ./logstash-filter-verifier --logstash-path=logstash test.json logstash.conf --loglevel=DEBUG 2016/02/16 13:39:44 Reading test case file: test.json (/Users/lukas/Projects/kb/logman/logstash-filter-verifier_1.0.1_linux_amd64/test.json) 2016/02/16 13:39:44 Starting "/usr/local/bin/logstash" with args ["logstash" "-w" "1" "--debug" "-e" "input { stdin { codec => \"json\" add_field => { } } } output { file { path => \"/var/folders/ny/rydql52n7qs32bh56v3dvdz00000gn/T/619215803\" codec => \"json_lines\" } }" "--log" "/var/folders/ny/rydql52n7qs32bh56v3dvdz00000gn/T/910667230" "--config" "logstash.conf"]. 2016/02/16 13:39:44 Waiting for child with pid 59259 to terminate. Error running Logstash: exit status 1. The process gave no output. ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 12:54pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/4 "2016-02-16T12:54:45Z")
Wait, what operating system is this? The path to your home directory indicates that it's Mac OS X, but you've downloaded a Linux release of LFV. ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 16, 2016, 2:15pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/5 "2016-02-16T14:15:04Z")
yeah, that's true, it's mac, and I confirm my mistake too the story is that I earlier downloaded the amd64 package and then realized my mistake so I changed just the binary with the one from darwin distribution ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 2:37pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/6 "2016-02-16T14:37:19Z")
So... you _were_ actually running the Darwin version of the file when you got this error? ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 16, 2016, 2:59pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/7 "2016-02-16T14:59:40Z")
yeps darwin binary, and before that I was working directly with fresh local make output of the github project basically when I run logstash with the args directly from the command line the logstash runs and outputs data to some var folder file. ```auto {"message":"test","tags":["_jsonparsefailure"],"@version":"1","@timestamp":"2016-02-16T14:57:28.985Z","host":"apulanta.local"} {"message":"test","tags":["_jsonparsefailure"],"@version":"1","@timestamp":"2016-02-16T14:57:29.928Z","host":"apulanta.local"} ``` Any reason for the tags \_jsonparsefailure? I am so excited to start using the tool so hopefully we are close the solution 🙂 ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 16, 2016, 3:22pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/8 "2016-02-16T15:22:01Z")
When the `message` field contains "test" a `_jsonparsefailure` tag is to be expected since the input string clearly isn't JSON. I can't reproduce what you describe on Linux: ```auto $ cat test.config filter { } $ cat test.json { "codec": "json", "ignore": ["host", "@timestamp"], "input": [ "{\"message\": \"This is a test message\"}" ], "expected": [ { "message": "This is a test message" } ] } $ logstash-filter-verifier --loglevel=DEBUG test.json test.config 2016/02/16 16:20:35 Reading test case file: test.json (/tmp/trash.s2LA/test.json) 2016/02/16 16:20:35 Starting "/opt/logstash/bin/logstash" with args ["/opt/logstash/bin/logstash" "-w" "1" "--debug" "-e" "input { stdin { codec => \"json\" add_field => { } } } output { file { path => \"/tmp/736256972\" codec => \"json_lines\" } }" "--log" "/tmp/145950395" "--config" "test.config"]. 2016/02/16 16:20:35 Waiting for child with pid 16737 to terminate. Comparing message 1 of test.json... 2016/02/16 16:20:39 Starting "/usr/bin/diff" with args ["-u" "/tmp/399511262/test.json/0/expected" "/tmp/399511262/test.json/0/actual"]. ``` I do have a Mac OS X machine at my disposal so I'll have to try it out there. ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 17, 2016, 8:28am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/9 "2016-02-17T08:28:32Z")
I have just tried on Linux machine, and the same result "The process gave no output.". ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 17, 2016, 8:32am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/10 "2016-02-17T08:32:48Z")
Did you try the exact commands I posted above? ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 17, 2016, 3:47pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/11 "2016-02-17T15:47:56Z")
yeps, everything as written few minutes ago we tried new vagrant machine with fresh unix, logstash 2.2.1 install and the logstash-filter-verifier binary worked there as expected thus there must be some differences which cause the no output error on those mac and unix machines I am using, I will try to hunt it ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 17, 2016, 3:49pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/12 "2016-02-17T15:49:23Z")
Thanks! I successfully ran this config on one machine yesterday but haven't had time to debug it further. ---
**Author:** ![zdenek.gallistl](https://avatars.discourse-cdn.com/v4/letter/z/b5e925/32.png) [@zdenek.gallistl](https://discuss.elastic.co/u/zdenek.gallistl)\ **Post date:** [February 18, 2016, 9:48am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/13 "2016-02-18T09:48:11Z")
Hi, I discussed with terramexx and found solution. In file /logstash-filter-verifier/logstash/process.go on line 82 is set one env variable for logstash process. All real evn variables are then lost including JAVA\_HOME. Logstahs than not able to find java. I changed this code c.Env = []string{"TZ=UTC"} by this: c.Env = os.Environ() c.Env = append(c.Env, "TZ=UTC") Also need to import "io" lib ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 18, 2016, 9:53am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/14 "2016-02-18T09:53:52Z")
Oh, good catch! Thanks! It was actually intentional to drop the current environment and only set TZ (so that tests are hermetic), but JAVA\_HOME should in that case also be propagated. LFV obviously needs to care about the stdout/stderr of the Logstash program and not just its log. ---
**Author:** ![terramexx](https://avatars.discourse-cdn.com/v4/letter/t/b9bd4f/32.png) [@terramexx](https://discuss.elastic.co/u/terramexx)\ **Post date:** [February 18, 2016, 11:02am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/15 "2016-02-18T11:02:15Z")
hooray, finally we can progress with test automation for logstash filters big thanks to both of you! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 22, 2016, 9:03pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/16 "2016-02-22T21:03:53Z")
Okay, so what I'm planning to do is: - Continue with a clean environment that only sets TZ=UTC but - add a `--keep-env` option that allows callers to specify the name of additional environment variables that should be propagated to the Logstash processes. Thereby you can run the tests like this: ``` logstash-filter-verifier --keep-env JAVA_HOME test.json test.config ``` If you _really_ want another TZ value you could override the default in the same manner. I'll release 1.1.0 with this change later this week. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 27, 2016, 7:38pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/17 "2016-02-27T19:38:34Z")
Logstash Filter Verifier 1.1.0 with the aforementioned change is now available: [https://github.com/magnusbaeck/logstash-filter-verifier/releases/tag/1.1.0](https://github.com/magnusbaeck/logstash-filter-verifier/releases/tag/1.1.0) ---
**Author:** ![csobchuk](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@csobchuk](https://discuss.elastic.co/u/csobchuk)\ **Post date:** [February 24, 2017, 9:46pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/18 "2017-02-24T21:46:45Z")
Presently getting the following failure when running make on Ubuntu 14.04 with build essentials, gdb, and golang installed: \>\> Setting [github.com/alecthomas/gometalinter](http://github.com/alecthomas/gometalinter) to version ca13be809964 \>\> Setting [github.com/alecthomas/kingpin](http://github.com/alecthomas/kingpin) to version v2.2.2 \>\> Setting [github.com/mattn/go-shellwords](http://github.com/mattn/go-shellwords) to version f4e566c536cf \>\> Setting [gopkg.in/matm/v1/gocov-html](http://gopkg.in/matm/v1/gocov-html) to version v1.0 \>\> Setting [github.com/mitchellh/packer](http://github.com/mitchellh/packer) to version v0.9.0 \>\> Setting [github.com/go-playground/overalls](http://github.com/go-playground/overalls) to version 845469c90499 \>\> Setting [github.com/op/go-logging](http://github.com/op/go-logging) to version dfaf3dff9b63 \>\> Setting [github.com/axw/gocov/gocov](http://github.com/axw/gocov/gocov) to version ac431cdb392e go build -o logstash-filter-verifier # [github.com/magnusbaeck/logstash-filter-verifier/logstash](http://github.com/magnusbaeck/logstash-filter-verifier/logstash) logstash/parallel\_process.go:57: ts.senderListener.SetUnlinkOnClose undefined (type \*net.UnixListener has no field or method SetUnlinkOnClose) make: \*\*\* [logstash-filter-verifier] Error 2 any ideas on what I have to do to correct it? TIA, ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 25, 2017, 10:18am UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/19 "2017-02-25T10:18:00Z")
You need Go 1.8, released last week. I'm going to add a note about this requirement to the readme file but haven't found the time. Sorry for the trouble. ---
**Author:** ![csobchuk](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@csobchuk](https://discuss.elastic.co/u/csobchuk)\ **Post date:** [February 27, 2017, 10:41pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880/20 "2017-02-27T22:41:28Z")
Thanks Magnus. It compiled once I installed 1.8 and removed the Ubuntu repository version of 1.2.1. However, now I get an error that logstash wasn't found. Using the example on your Github page for the syslog.json and test.config I ran : ./logstash-filter-verifier syslog.json test.config and received the following output. Running tests in syslog.json... fork/exec /opt/logstash/bin/logstash: no such file or directory Logstash 5.2 for Ubuntu 14.04, at least, is in /usr/share/logstash on Linux. I suspect I have to change this in the .go file and then recompile? It seems there is no way to discover this from the environment? Am I on the right track to fix this? I did download it for windows as well - which failed for the same reason. [Next page](https://discuss.elastic.co/t/logstash-filter-verifier-any-working-example/41880.md?page=2)