Logstash pipeline running but not loading the data

Elastic Stack Logstash
1

Hello! I'm starting to learn Elastick Stack. Right now I'm trying to load some data from a csv file from a tutorial. I think everything is ok, the pipeline is running but no data is loaded to Elastic.

This is the conf file:

input {
  file {
    path => "C:\temp\Elastic_stack\files\products.csv"
    start_position => "beginning"
    sincedb_path => "NULL" 
    
    codec => plain {
       charset => "ISO-8859-1"
    }
  
}
filter {
  csv {
    separator => ","
    columns => ["id","title","description","manufacturer","price"]
  }

  mutate {
    remove_field => ["@version","@timestamp","path","host", "tags", "message"]
  } 
}
output {
  elasticsearch {
    hosts => "https://localhost:9200/"
    user => "elastic"
    password => "xxxxxx"
    index => "test_products"
    document_type => "products"
    cacert => 'C:\temp\Elastic_stack\kibana-8.1.1\data\ca_1648728207940.crt'
    ssl => true
    ssl_certificate_verification => false
  }
  stdout {}
}

And I getting this output (with --log.level debug) :

[2022-04-03T10:22:11,454][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.04}
[2022-04-03T10:22:11,513][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-04-03T10:22:11,527][DEBUG][logstash.javapipeline    ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x960e888 run>"}
[2022-04-03T10:22:11,528][DEBUG][org.logstash.execution.PeriodicFlush][main] Pushing flush onto pipeline.
[2022-04-03T10:22:11,559][INFO ][filewatch.observingtail  ][main][d9d4b26bb2c567c8c8b7cb6af0f92c3de795d23b89fc1d1b0f5f8ec86d131942] START, creating Discoverer, Watch with file and sincedb collections
[2022-04-03T10:22:11,582][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-04-03T10:22:11,592][DEBUG][filewatch.sincedbcollection][main][d9d4b26bb2c567c8c8b7cb6af0f92c3de795d23b89fc1d1b0f5f8ec86d131942] open: reading from NULL
[2022-04-03T10:22:12,670][DEBUG][filewatch.sincedbcollection][main][d9d4b26bb2c567c8c8b7cb6af0f92c3de795d23b89fc1d1b0f5f8ec86d131942] writing sincedb (delta since last write = 1648992132)
[2022-04-03T10:22:14,163][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2022-04-03T10:22:15,008][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2022-04-03T10:22:15,010][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}
[2022-04-03T10:22:16,536][DEBUG][org.logstash.execution.PeriodicFlush][main] Pushing flush onto pipeline.
[2022-04-03T10:22:19,178][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu
[2022-04-03T10:22:20,057][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ParNew"}
[2022-04-03T10:22:20,057][DEBUG][logstash.instrument.periodicpoller.jvm] collector name {:name=>"ConcurrentMarkSweep"}

I think the problem is the files or directories not founded but I don't find any more info to fix this:
One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

any help are welcome!

2

Do not use backslash in the path option of a file input. It is treated as an escape. Change them to forward slash, or \\.

Also, setting the sincedb_path to NULL will persist the in-memory sincedb to a file called NULL in the working directory of logstash. If you do not want the in-memory db persisted across restarts then set it to "NUL"

3

Thanks! I changed to forward slash and it's working! :man_facepalming:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.