# Logstash issue with Netflow module **URL:** \ **Category:** Logstash\ **Created:** [October 5, 2017, 7:16pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898 "2017-10-05T19:16:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 5, 2017, 7:16pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/1 "2017-10-05T19:16:32Z")
HI : 😄 I'm having this issue with logstash with netflow module active: If I ran it from CLI with this command : ./logstash --modules netflow It work perfect , Kibana get the index patter netflow-\* and I can see populated graphs , etc, etc... Now if I start it with systemctl : $systemctl start logstash.service Them kibana can't find the index pattern . regards JC ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 5, 2017, 7:51pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/2 "2017-10-05T19:51:35Z")
You must first run it with the `--setup` flag to populate the dashboards and such. After running it with `--setup` once, you won't have to run it with that again, unless you need to update those dashboards/visualizations. ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 5, 2017, 8:11pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/3 "2017-10-05T20:11:21Z")
# I did ran it with --setup but as soon I restart logstash.service , kibana can't find netflow-\* index. Them if I ran it from terminal ./logstash --modules netflow, everthing work perfect . The issue is when I start it from $systemctl like this: # $systemctl start logstash.service If I run $systemctl status logstash : I got that logstash is green but is not populating indexes in ES and Kibana ask me set a new index pattern. I know this caz : this is the output of : curl -X GET '[http://localhost:9200/\_cat/indices?v](http://localhost:9200/_cat/indices?v)' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana t4M9dOX0T9WuSyeAYgNbZA 1 1 1 0 3.2kb 3.2kb not logstash indexes.. regards and thank you for helping me JC ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 5, 2017, 8:32pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/4 "2017-10-05T20:32:13Z")
and this is the output when I ran logstash from Terminal passing the --modules netflow. :~$ curl -X GET '[http://localhost:9200/\_cat/indices?v](http://localhost:9200/_cat/indices?v)' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open .kibana t4M9dOX0T9WuSyeAYgNbZA 1 1 89 1 101.1kb 101.1kb yellow open netflow-2017.10.05 r52HE47mTIGdl6wfiR2yQQ 5 1 24725 0 17.4mb 17.4mb them you can see netflow index is populating elasticS ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 3:12pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/5 "2017-10-06T15:12:21Z")
nobody? 😞 ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 4:08pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/6 "2017-10-06T16:08:00Z")
> [@juanquy](#): > > Now if I start it with systemctl : > > $systemctl start logstash.service > > Them kibana can't find the index pattern . This is a red herring. Kibana and Logstash have no direct ties to each other. If the index pattern was set once, it's there forever (or until you manually delete it). It doesn't go away. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 4:10pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/7 "2017-10-06T16:10:06Z")
The real issue is that you have to populate `logstash.yml` with the proper `modules` configuration for it to be picked up when started with `systemctl`. There are 2 ways to tell Logstash to use the module configuration: 1. Using `--modules` on the command-line 2. Configuring the `modules` section in `logstash.yml` I'm guessing you didn't configure `logstash.yml` to do modules. Mine looks like this: ```auto modules: - name: netflow var.elasticsearch.hosts: "esclient.REDACTED.net" var.elasticsearch.username: "MY_XPACK_USER" var.elasticsearch.password: "REDACTED" var.input.udp.port: 2055 var.kibana.host: "kibana.REDACTED.net:5601" var.kibana.username: "MY_XPACK_USER" var.kibana.password: "REDACTED" ``` If you're not using X-Pack, then you can omit the username and password lines. ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 6:09pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/8 "2017-10-06T18:09:47Z")
Here is my logstash.yml, you may find the issue. ```auto ======================================== # # ------------ Module Settings --------------- # Define modules here. Modules definitions must be defined as an array. # The simple way to see this is to prepend each `name` with a `-`, and keep # all associated variables under the `name` they are associated with, and # above the next, like this: # modules: - name: netflow var.elasticsearch.host: "localhost" var.input.udp.port: 2055 var.kibana.host: "localhost:5601" # var.PLUGINTYPE1.PLUGINNAME1.KEY2: VALUE # var.PLUGINTYPE2.PLUGINNAME1.KEY1: VALUE # var.PLUGINTYPE3.PLUGINNAME3.KEY1: VALUE # # Module variable names must be in the format of # # var.PLUGIN_TYPE.PLUGIN_NAME.KEY # # modules: # # ------------ Queuing Settings -------------- ``` ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 7:36pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/9 "2017-10-06T19:36:17Z")
Please encapsulate pasted configurations between triple-backticks, like this: ```` ``` YOUR CONFIG HERE ``` ```` I took the liberty of adding those to what you pasted, as the `#` key makes a header in markdown. It was impossible to tell whether things were properly configured or not. This configuration looks fine. Have you checked to see if the port is open after Logstash starts? ```auto netstat -na | grep 2055 ``` ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 7:40pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/10 "2017-10-06T19:40:20Z")
Oh I will encapsulate it. netstat -na | grep 2055 not showing any output. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 7:56pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/11 "2017-10-06T19:56:05Z")
That implies that Logstash isn't running the way you want. It could be permissions related, as you launched it (perhaps as root?) as a different user first, and then tried to run it using `systemctl`. But without looking at the logs, it would be hard to say. Is logstash actually running? ```auto systemctl status logstash ``` Have you looked at the logs in `/var/log/logstash`? ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 8:01pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/12 "2017-10-06T20:01:53Z")
I will look at the logs really did not think about it. yes I do start the services with systemctl I'm using Ubuntu Server 16.04 The service is active and green. ... systemctl status logstash ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: Active: active (running) since Fri 2017-10-06 14:59:12 CDT; 12s ago Main PID: 18113 (java) Tasks: 18 Memory: 344.9M CPU: 36.418s CGroup: /system.slice/logstash.service └─18113 /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CM Oct 06 14:59:12 ELK systemd[1]: Started logstash. ... ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 8:06pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/13 "2017-10-06T20:06:58Z")
wierd! I do have logs from yesterday but not from today! ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/14 "2017-10-06T20:07:48Z")
Try shutting down the service and then starting (not `restart`). Something is not starting correctly, and a fresh start is in order. Look for the logs at that point. ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 8:09pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/15 "2017-10-06T20:09:11Z")
I do not have to config any pipeline since I'm using the module right? the module will be the main pipeline in this case right? ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 8:09pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/16 "2017-10-06T20:09:42Z")
I will thanks ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 8:10pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/17 "2017-10-06T20:10:39Z")
> [@juanquy](#): > > I do not have to config any pipeline since I'm using the module right? the module will be the main pipeline in this case right? That is correct. ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 8:31pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/18 "2017-10-06T20:31:24Z")
not a clue, I think I been messing a lot with this now something is wrong when I start LS via terminal : ... ./logstash --modules netflow --path.settings /etc/logstash it start but 10 sec after stop.. that was not happening before... ussually start . what about .... apt-get purge logstash ... and start fresh ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 6, 2017, 8:33pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/19 "2017-10-06T20:33:53Z")
> [@juanquy](#): > > it start but 10 sec after > > stop.. Why does it stop? This is important. Understanding the why will prevent it from happening again, even after purging and reinstalling. What is in the logs? Something should be logged after that. If there's a failure, it will log something. ---
**Author:** ![juanquy](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanquy](https://discuss.elastic.co/u/juanquy)\ **Post date:** [October 6, 2017, 9:05pm UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898/20 "2017-10-06T21:05:38Z")
I will post the logs [Next page](https://discuss.elastic.co/t/logstash-issue-with-netflow-module/102898.md?page=2)