Logstash does not create index in Elasticsearch (Windows 10)

[Lucas_Yoshioka]

Hi guys, I am trying run ELK, but for some reason Logstash does not create my indexes in Elasticsearch. I installed both and added their /bin path to my environment variables.

Here is the conf file I am using:

input {
  file {
    type=>"user"
    path=>"C:/Users/lucas/projects/centralizedlogs/centralizedlogs.log"
    sincedb_path=>"NUL"
  }
}

output {
  if [type] == "user" {
    elasticsearch {
      hosts => ["localhost:9200"]
      index => "user"
    }
  }

  stdout { codec => rubydebug }
}

Log on Windows Power Shell

PS C:\Users\lucas\Documents\ELK\logstash-7.13.4> .\bin\logstash -f .\simple-config.conf
"Using bundled JDK: ""
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to C:/Users/lucas/Documents/ELK/logstash-7.13.4/logs which is now configured via log4j2.properties
[2021-07-28T01:57:57,246][INFO ][logstash.runner          ] Log4j configuration path used is: C:\Users\lucas\Documents\ELK\logstash-7.13.4\config\log4j2.properties
[2021-07-28T01:57:57,256][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.13.4", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [mswin32-x86_64]"}
[2021-07-28T01:57:57,393][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-07-28T01:57:58,162][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-07-28T01:57:58,768][INFO ][org.reflections.Reflections] Reflections took 39 ms to scan 1 urls, producing 24 keys and 48 values
[2021-07-28T01:58:00,054][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2021-07-28T01:58:00,416][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2021-07-28T01:58:00,583][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2021-07-28T01:58:00,630][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (7.13.4) {:es_version=>7}
[2021-07-28T01:58:00,633][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2021-07-28T01:58:00,748][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:disabled}
[2021-07-28T01:58:00,792][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/lucas/Documents/ELK/logstash-7.13.4/simple-config.conf"], :thread=>"#<Thread:0x54502a6c run>"}
[2021-07-28T01:58:01,671][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.88}
[2021-07-28T01:58:02,044][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-07-28T01:58:02,093][INFO ][filewatch.observingtail  ][main][f6a0285c9af610842972cd826eb29ba37ee51be952b8cfe59689387dd3d6356f] START, creating Discoverer, Watch with file and sincedb collections
[2021-07-28T01:58:02,105][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

[warkolm (Mark Walkom)]

Welcome to our community! We aren't all guys though.

You can try running Logstash with debug on to see what's happening. But it's likely the sincedb.
Which means you can try using type to echo the file into Logstash with a stdin input section and see if that works.

[leandrojmp (Leandro Pereira)]

Is the file being written or it is not being written anymore?

You didn't set start_position in the file input, so logstash per default will read the file from the end, which means that it will only start reading the file when new lines arrive.

If you want to read from the start you need to use start_position => "beginning" in the file input.