I am trying to send some log files to elastic search directly using filebeat.
I am not getting any errors in filebeat or elasticsearch.
Below is the filebeat config:
filebeat log:
2017-10-24T13:39:05-05:00 INFO Home path: [C:\Program Files\filebeat-5.5.2-windows-x86_64] Config path: [C:\Program Files\filebeat-5.5.2-windows-x86_64] Data path: [C:\ProgramData\filebeat] Logs path: [C:\Program Files\filebeat-5.5.2-windows-x86_64\logs]
2017-10-24T13:39:05-05:00 INFO Metrics logging every 30s
2017-10-24T13:39:05-05:00 INFO Setup Beat: filebeat; Version: 5.5.2
2017-10-24T13:39:05-05:00 INFO Loading template enabled. Reading template file: C:\Program Files\filebeat-5.5.2-windows-x86_64\filebeat.template.json
2017-10-24T13:39:05-05:00 INFO Loading template enabled for Elasticsearch 2.x. Reading template file: C:\Program Files\filebeat-5.5.2-windows-x86_64\filebeat.template-es2x.json
2017-10-24T13:39:05-05:00 INFO Loading template enabled for Elasticsearch 6.x. Reading template file: C:\Program Files\filebeat-5.5.2-windows-x86_64\filebeat.template-es6x.json
2017-10-24T13:39:05-05:00 INFO Elasticsearch url: http://localhost:9200
2017-10-24T13:39:05-05:00 INFO Activated elasticsearch as output plugin.
2017-10-24T13:39:05-05:00 INFO Publisher name: VW24998
2017-10-24T13:39:05-05:00 INFO Flush Interval set to: 1s
2017-10-24T13:39:05-05:00 INFO Max Bulk Size set to: 50
2017-10-24T13:39:05-05:00 INFO filebeat start running.
2017-10-24T13:39:05-05:00 INFO Registry file set to: C:\ProgramData\filebeat\registry
2017-10-24T13:39:05-05:00 INFO Loading registrar data from C:\ProgramData\filebeat\registry
2017-10-24T13:39:05-05:00 INFO States Loaded from registrar: 75
2017-10-24T13:39:05-05:00 INFO Loading Prospectors: 1
2017-10-24T13:39:05-05:00 INFO Starting Registrar
2017-10-24T13:39:05-05:00 INFO Start sending events to output
2017-10-24T13:39:05-05:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017-10-24T13:39:05-05:00 INFO Prospector with previous states loaded: 47
2017-10-24T13:39:05-05:00 INFO Starting prospector of type: log; id: 13285866520393955790
2017-10-24T13:39:05-05:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2017-10-24T13:39:35-05:00 INFO Non-zero metrics in the last 30s: publish.events=47 registrar.states.current=75 registrar.states.update=47 registrar.writes=1
2017-10-24T13:40:05-05:00 INFO No non-zero metrics in the last 30s
2017-10-24T13:40:35-05:00 INFO No non-zero metrics in the last 30s
2017-10-24T13:41:05-05:00 INFO No non-zero metrics in the last 30s
2017-10-24T13:41:35-05:00 INFO No non-zero metrics in the last 30s
2017-10-24T13:42:05-05:00 INFO No non-zero metrics in the last 30s
I have kept elastic search running -
ElasticSearch Log:
[2017-10-24T10:13:42,700][INFO ][o.e.n.Node ] [MyNode] initializing ...
[2017-10-24T10:13:45,081][INFO ][o.e.e.NodeEnvironment ] [MyNode] using [1] data paths, mounts [[(C:)]], net usable_space [101.6gb], net total_space [462.9gb], spins? [unknown], types [NTFS]
[2017-10-24T10:13:45,083][INFO ][o.e.e.NodeEnvironment ] [MyNode] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-10-24T10:13:45,087][INFO ][o.e.n.Node ] [MyNode] node name [MyNode], node ID [z8TE95jFR6KD_gFQZRojiQ]
[2017-10-24T10:13:45,088][INFO ][o.e.n.Node ] [MyNode] version[5.5.2], pid[8364], build[b2f0c09/2017-08-14T12:33:14.154Z], OS[Windows 7/6.1/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_144/25.144-b01]
[2017-10-24T10:13:45,090][INFO ][o.e.n.Node ] [MyNode] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Delasticsearch, -Des.path.home=C:\Program Files\elasticsearch-5.5.2]
[2017-10-24T10:13:53,244][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [aggs-matrix-stats]
[2017-10-24T10:13:53,245][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [ingest-common]
[2017-10-24T10:13:53,246][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [lang-expression]
[2017-10-24T10:13:53,248][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [lang-groovy]
[2017-10-24T10:13:53,249][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [lang-mustache]
[2017-10-24T10:13:53,250][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [lang-painless]
[2017-10-24T10:13:53,251][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [parent-join]
[2017-10-24T10:13:53,252][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [percolator]
[2017-10-24T10:13:53,253][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [reindex]
[2017-10-24T10:13:53,255][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [transport-netty3]
[2017-10-24T10:13:53,256][INFO ][o.e.p.PluginsService ] [MyNode] loaded module [transport-netty4]
[2017-10-24T10:13:53,257][INFO ][o.e.p.PluginsService ] [MyNode] no plugins loaded
[2017-10-24T10:14:21,629][INFO ][o.e.d.DiscoveryModule ] [MyNode] using discovery type [zen]
[2017-10-24T10:14:25,089][INFO ][o.e.n.Node ] [MyNode] initialized
[2017-10-24T10:14:25,091][INFO ][o.e.n.Node ] [MyNode] starting ...
[2017-10-24T10:14:28,600][INFO ][o.e.t.TransportService ] [MyNode] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2017-10-24T10:14:32,193][INFO ][o.e.c.s.ClusterService ] [MyNode] new_master {MyNode}{z8TE95jFR6KD_gFQZRojiQ}{F26xRT36SQuGWYIw3t1ZVw}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-10-24T10:14:32,671][INFO ][o.e.g.GatewayService ] [MyNode] recovered [0] indices into cluster_state
[2017-10-24T10:14:33,285][INFO ][o.e.h.n.Netty4HttpServerTransport] [MyNode] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2017-10-24T10:14:33,286][INFO ][o.e.n.Node ] [MyNode] started
I am unable to figure out what is going wrong here and why the index is not getting created.