Http poller illegal argument exception

Elastic Stack Logstash
1

I am trying to use the http poller to pull eia.gov.
U.S. Energy Information Administration - EIA - Independent Statistics and Analysis

I'm doing the series query and it is a json output.

It is attempting to pull the json output but it's getting stuck on the series.data field there is a string and a float as detailed below.

data": [
[
"201605",
41.02
],
[
"201604",
35.59
],
[
"201603",
31.87
],

my question is, how do i get logstash to properly handle this? would this require a mutate? could I break it up into another field (series.data.201608: float? I think once I overcome this obstacle I'll be in the clear.

2

So you want to separate the strings and floats into separate fields? You'll have to use a ruby filter for this.

3

Well, I'm really just looking to get around the problem of having the string and float in the same field. I'm not really sure what the best way to go about doing it. i was thinking either making them separate fields or making the string a field under data.

Option 1:

"data": [
"201605": 41.02,
"201604": 35.59,
"201603": 31.87
],

Option 2:

"data":[
"date": "201605",
"price": 41.02
],
[
"date": "201604",
"price": 35.59
],
[
"date": "201603",
"price": 31.87
],

If that helps clear up what I'm envisioning to resolve this issue.

4

so i just wanted to come back to this issue as i have not been able to resolve it.

i've been trying to use filters to fix it. i've tried splitting and it failed, i've tried converting it to a different datatype and that failed, i tried removing the double quotes and that did not help either.

The only way i got logstash running with this input was to remove the field entirely via filter... doesn't anyone have a recommended filter config i could try as a sanity check?