When specifying the schema for the logstash-output-google_bigquery plugin whenever i select integer or float as the type for the field i get a cannot convert value to integer. This problem has been pinned down to having empty fields. is there a way to allow empty fields to be uploaded?
BQ: job failed, please enable debug and check full response (probably the issue is an incompatible schema). NOT deleting local file. {:job_id=>"job_Yz72z1YtMQ56SdH4-4S1SQYiFlM", :filename=>"/tmp/logstash-bq1/logstash_bq_etl-log-guc-01_2016-07-15T18:00.part051.log", :job_status=>{"state"=>"DONE", "errorResult"=>{"reason"=>"invalid", "location"=>"file-00000000", "message"=>"JSON table encountered too many errors, giving up. Rows: 1; errors: 1."}, "errors"=>[{"reason"=>"invalid", "location"=>"file-00000000", "message"=>"JSON table encountered too many errors, giving up. Rows: 1; errors: 1."}, {"reason"=>"invalid", "message"=>"JSON parsing error in row starting at position 0 at file: file-00000000. Could not convert value to integer. Field: headAdvertiserId; Value: "}]}, :level=>:error}
This seems more like a BigQuery question.
yes sorry i found it was an error on the big query side due to my logstash fields being set to empty strings so as to avoid having the field names printed. but thank you for the help