File input not working

[cowensel (Peter Cowen)]

I am looking to use logstash to ingest a csv file that contains some stats and then graph them. I have setup a file input but I am unable to get it to work.

My logstash config is
input {
file {
path => "C:\logstash-6.4.2\files*.csv"
start_position => "beginning"
}
}
filter {
csv {
columns => ["Action","Apdex","Count","Avg (ms)","SD (ms)","Min (ms)","Max (ms)","Total (ms)","Total (% time)","Dissat (%)"]
separator => ","
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "stats-%{+YYYY.MM.dd}"
template_overwrite => true
}
stdout { codec => rubydebug }
}

am I doing something incorrectly?

[AquaX (Andreas Helmer)]

Are you getting any error messages?

[cowensel (Peter Cowen)]

unfortunately not.

the logstash logs show logstash starts ok but it does not process anything and i'm unsure why.

[AquaX (Andreas Helmer)]

Then it's most likely one of two things:
Your path does not contain the files that match your path pattern.
-Check your path and files to make sure that they are ok.
OR
The sincedb is showing that all files have been processed
-Delete the .sincedb file and restart logstash, if that works then you'll need to add some additional parameters to your config like setting the sincedb_path to NUL.

[cowensel (Peter Cowen)]

Following the steps above setting the sincedb_path to "NuLL" and checking the files all seemed ok. I then placed the logstash logs in debug I received the following in a loop
2018-12-02T22:59:01,649][DEBUG][logstash.instrument.periodicpoller.cgroup] One or more required cgroup files or directories not found: /proc/self/cgroup, /sys/fs/cgroup/cpuacct, /sys/fs/cgroup/cpu

Downgrading to ELK 6.0.0 resolved the problem but now the csv has now uploaded ok.

[cowensel (Peter Cowen)]

I have uploaded the file ok and I have added a mutate convert to the config but when imported it makes the columns 0 .
The file has columns like col1:16591.07535,col 2: 2310, col3: 7950,col4: 1.5, col5: 180000, col6: 38300000
but when imported to elastic it shows 0.
The mutate section of config is
convert => {
"Apdex" => "integer"
"Count" => "integer"
"Avg (ms)" => "integer"
"SD (ms)" => "integer"
"Min (ms)" => "integer"
"Max (ms)" => "integer"
"Total (ms)" => "integer"
"Total (% time)" => "integer"
"Dissat (%)" => "integer"
}
What am I doing incorrectly?

[balumurari1]

Hello,
Make sure to change the path from
path => "C:\logstash-6.4.2\files*.csv"

to

path => "C:/logstash-6.4.2/files*.csv"

Hope this helps you,

Load csv file in logstash

Thanks for reply,hope this works fine, input { file { path => "D:/Balu/ELK-stack/csvfile.csv" start_position => "beginning" sincedb_path => "NUL" } } filter { csv { separator => "," columns => ["open","high","low","close","volume"] } } output { elasticsearch { hosts => "http://localhost:9200" index => "wallet-address-index" } stdout { codec => rubydebug } }