Invalid field format when writing data to influxdb from csv file using logstash-output-plugin


#1

Hi there!

I am trying to get vmstat output (in CSV format) and load the data into influxdb using logstash and logstash-output-plugin, but am getting error "invalid field format" when writing data to influxdb. See details below.

Any assistance would be much appreciated.

Thanks

OS: Windows 7 Enterprise
InfluxDB: 1.5.2-1
logstash: 6.2.4
logstash-output-influxdb plugin

CSV file:
2018-06-14 12:09:30.123,13,0,0
2018-06-14 12:09:40.123,27,0,0
2018-06-14 12:09:50.123,98,0,0

logstash config file:
input {
stdin {}
file {
path => "C:/Temp/*.csv"
start_position => "beginning"
}
}

filter {
csv {
separator => ","
columns => ["vmtimestamp","procsR","procsB","procsW"]

	convert => {
	  "procsR" => "integer"
	  "procsB" => "integer"
	  "procsW" => "integer"
	}
}

}

output {
influxdb {
id => "influxdb1"
host => "localhost"
db => "monitoring"
allow_time_override => true
exclude_fields => ["@timestamp", "@version", "sequence", "message", "type", "path"]
codec => json
}

stdout { codec => json}
}

logstash stdout:
The stdin plugin is now waiting for input:
[2018-06-14T14:55:10,886][INFO ][logstash.agent ] Pipelines running {:
count=>1, :pipelines=>["main"]}
[2018-06-14T14:55:11,870][WARN ][logstash.outputs.influxdb] Non recoverable exce
ption while writing to InfluxDB {:exception=>#<InfluxDB::Error: {"error":"unable
to parse 'logstash 1528952111126': invalid field format"}
}
{"procsR":13,"procsW":0,"@timestamp":"2018-06-14T04:55:11.126Z","@version":"1","
path":"C:/Temp/oncillat_vmstat_201806141454.csv","message":"2018-06-14 12:09:30.
123,13,0,0","procsB":0,"vmtimestamp":"2018-06-14 12:09:30.123","host":"NIGSA9370
04"}{"procsR":98,"procsW":0,"@timestamp":"2018-06-14T04:55:11.151Z","@version":"
1","path":"C:/Temp/oncillat_vmstat_201806141454.csv","message":"2018-06-14 12:09
:50.123,98,0,0","procsB":0,"vmtimestamp":"2018-06-14 12:09:50.123","host":"NIGSA
937004"}{"procsR":27,"procsW":0,"@timestamp":"2018-06-14T04:55:11.150Z","@versio
n":"1","path":"C:/Temp/oncillat_vmstat_201806141454.csv","message":"2018-06-14 1
2:09:40.123,27,0,0","procsB":0,"vmtimestamp":"2018-06-14 12:09:40.123","host":"N
IGSA937004"}[2018-06-14T14:55:12,093][WARN ][logstash.outputs.influxdb] Non reco
verable exception while writing to InfluxDB {:exception=>#<InfluxDB::Error: {"er
ror":"unable to parse 'logstash 1528952111150': invalid field format\nunable to
parse 'logstash 1528952111151': invalid field format"}

}

influxdb log:
[httpd] 127.0.0.1 - - [14/Jun/2018:14:55:11 +1000] "POST /write?db=monitoring&pr
ecision=ms&rp=autogen HTTP/1.1" 400 76 "-" "Ruby" 1e7cd7d8-6f8f-11e8-8029-000000
000000 2000
[httpd] 127.0.0.1 - - [14/Jun/2018:14:55:12 +1000] "POST /write?db=monitoring&pr
ecision=ms&rp=autogen HTTP/1.1" 400 141 "-" "Ruby" 1eb32a18-6f8f-11e8-802a-00000
0000000 1000


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.