Configuring Input file on Windows to get Output in 2024

I am trying a simple file input / output example. Using following conf file at the C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/config/file-pipeline.conf

input {
    file {
        path => ["C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/logstash-tutorial-dataset.txt"]
        start_position => "beginning"
        ignore_older => 0  
output {
    stdout {}
    file {
        path => "C:/output.txt"

running as

C:\logstash-8.11.3-windows-x86_64\logstash-8.11.3>.\bin\logstash.bat -f config\file-pipeline.conf --debug --verbose

I have verified that the C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/logstash-tutorial-dataset.txt exist.

However, no matter how I try. I am not seeing anything in the expected C:/output.txt file. What am I missing?

Below is what I see

[2024-01-25T22:55:07,492][INFO ][logstash.codecs.jsonlines] ECS compatibility is enabled but `target` option was not specified. This may cause fields to be set at the top-level of the event where they are likely to clash with the Elastic Common Schema. It is recommended to set the `target` option to avoid potential schema conflicts (if your data is ECS compliant or non-conflicting, feel free to ignore this message)
[2024-01-25T22:55:07,514][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2024-01-25T22:55:07,548][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/config/file-pipeline.conf"], :thread=>"#<Thread:0x5e723db3 C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2024-01-25T22:55:08,183][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.63}
[2024-01-25T22:55:08,198][INFO ][logstash.inputs.file     ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/data/plugins/inputs/file/.sincedb_9ce64e16f0fb3d7c0a02a88e872cfdfd", :path=>["C:/logstash-8.11.3-windows-x86_64/logstash-8.11.3/logstash-tutorial-dataset.txt"]}
[2024-01-25T22:55:08,201][INFO ][logstash.javapipeline    ][main] Pipeline started {""=>"main"}
[2024-01-25T22:55:08,221][INFO ][filewatch.observingtail  ][main][4d54d35412a212527381f3b4bf324aee77aa50c813451870a27852af4c3f2a90] START, creating Discoverer, Watch with file and sincedb collections
[2024-01-25T22:55:08,232][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}

See this thread.

Thanks. It looks like understanding ignore_older solved this problem.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.