Sorry I couldn't fit the whole update into the past reply.
But there are some updates.
I literally dropped the old environment and set up a new one with the editions on logstash.conf and a new filebeat.yml
- type: log
enabled: true
paths:
- /var/log/syslog
#- type: log
# enabled: true
# paths:
# - /var/log/openvpn/*.csv
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.kibana:
host: "192.168.1.23:5601"
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["192.168.1.23:9200"]
# Protocol - either `http` (default) or `https`.
#protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "elastic"
password: "secret"
The filebeat log after running filebeat setup -e
2021-04-20T17:37:05.705-0400 INFO instance/beat.go:668 Beat ID: 31c85ee3-f3e0-4140-ad1f-4f6611c93bf5
2021-04-20T17:37:05.705-0400 INFO [beat] instance/beat.go:996 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "31c85ee3-f3e0-4140-ad1f-4f6611c93bf5"}}}
2021-04-20T17:37:05.705-0400 INFO [beat] instance/beat.go:1005 Build info {"system_info": {"build": {"commit": "08e20483a651ea5ad60115f68ff0e53e6360573a", "libbeat": "7.12.0", "time": "2021-03-18T06:16:51.000Z", "version": "7.12.0"}}}
2021-04-20T17:37:05.705-0400 INFO [beat] instance/beat.go:1008 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.15.8"}}}
2021-04-20T17:37:05.706-0400 INFO [beat] instance/beat.go:1012 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-04-09T18:02:13-04:00","containerized":false,"name":"vpn","ip":["127.0.0.1/8","::1/128","192.168.1.22/24","fe80::5054:ff:fe96:8be4/64","172.17.0.1/16","fe80::42:65ff:fee8:707a/64","fe80::acaf:4eff:fe52:555d/64","192.168.85.1/24","fe80::75ec:f495:858c:a045/64"],"kernel_version":"4.15.0-140-generic","mac":["52:54:00:96:8b:e4","02:42:65:e8:70:7a","ae:af:4e:52:55:5d"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.5 LTS (Bionic Beaver)","major":18,"minor":4,"patch":5,"codename":"bionic"},"timezone":"-04","timezone_offset_sec":-14400,"id":"2c43069cfd43444e98e47c2020bbb338"}}}
2021-04-20T17:37:05.707-0400 INFO [beat] instance/beat.go:1041 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/home/producao", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 4497, "ppid": 4496, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2021-04-20T17:37:04.950-0400"}}}
2021-04-20T17:37:05.708-0400 INFO instance/beat.go:304 Setup Beat: filebeat; Version: 7.12.0
2021-04-20T17:37:05.709-0400 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'filebeat-7.12.0' as ILM is enabled.
2021-04-20T17:37:05.709-0400 INFO eslegclient/connection.go:99 elasticsearch url: http://elasticsearch:9200
2021-04-20T17:37:05.709-0400 INFO [publisher] pipeline/module.go:113 Beat name: vpn
2021-04-20T17:37:05.711-0400 INFO eslegclient/connection.go:99 elasticsearch url: http://elasticsearch:9200
2021-04-20T17:37:05.870-0400 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.12.0
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.
2021-04-20T17:37:05.901-0400 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2021-04-20T17:37:05.904-0400 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
2021-04-20T17:37:05.904-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
2021-04-20T17:37:05.904-0400 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{filebeat-7.12.0 {now/d}-000001}' as ILM is enabled.
2021-04-20T17:37:05.904-0400 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'filebeat-7.12.0-*' as ILM is enabled.
2021-04-20T17:37:05.904-0400 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.12.0 {now/d}-000001} as ILM is enabled.
2021-04-20T17:37:05.904-0400 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2021-04-20T17:37:05.908-0400 INFO template/load.go:183 Existing template will be overwritten, as overwrite is enabled.
2021-04-20T17:37:08.178-0400 INFO template/load.go:117 Try loading template filebeat-7.12.0 to Elasticsearch
2021-04-20T17:37:08.684-0400 INFO template/load.go:109 template with name 'filebeat-7.12.0' loaded.
2021-04-20T17:37:08.684-0400 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
2021-04-20T17:37:08.687-0400 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2021-04-20T17:37:08.688-0400 INFO kibana/client.go:119 Kibana url: http://elasticsearch:5601
It appeared the new filebeat-* index pattern and it is logging from the client var/log/syslog
I don't know if it was intended, now the logstash service isn't running.
[2021-04-20T21:30:18,618][ERROR][logstash.javapipeline ][main][6343e707a843e0a33ea4e9006c0672156e8ebce3e38ad7bbab2f83fed8b4dfeb] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Beats port=>5044, id=>"6343e707a843e0a33ea4e9006c0672156e8ebce3e38ad7bbab2f83fed8b4dfeb", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_810f74c2-2bbf-4def-928e-d4f6a4fc9027", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl=>false, add_hostname=>false, ssl_verify_mode=>"none", ssl_peer_metadata=>false, include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384", "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256"], client_inactivity_timeout=>60, executor_threads=>4>
Error: Address already in use
Exception: Java::JavaNet::BindException
Stack: sun.nio.ch.Net.bind0(Native Method)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:455)
sun.nio.ch.Net.bind(sun/nio/ch/Net.java:447)
sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:227)
io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:134)
io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:550)
io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1334)
io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:506)
io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:491)
io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:973)
io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:248)
io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:356)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:164)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:472)
io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:500)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:989)
io.netty.util.internal.ThreadExecutorMap$2.run(io/netty/util/internal/ThreadExecutorMap.java:74)
io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
java.lang.Thread.run(java/lang/Thread.java:834)
[2021-04-20T21:30:18,874][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2021-04-20T21:30:18,888][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2021-04-20T21:30:19,621][INFO ][org.logstash.beats.Server][main][6343e707a843e0a33ea4e9006c0672156e8ebce3e38ad7bbab2f83fed8b4dfeb] Starting server on port: 5044
[2021-04-20T21:30:20,109][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
[2021-04-20T21:30:23,986][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2021-04-20T21:30:24,003][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@localhost:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elastic:xxxxxx@localhost:9200/][Manticore::SocketException] Connection refused (Connection refused)"}
[2021-04-20T21:30:25,332][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
And connection to 5044 port is refused.
Thanks in advance.
