Vega Error

Krunal · May 31, 2021, 9:25am

Hello expert,
I need your help to find the error, i tried to find out but i didnt got it. Thanks in advance...

` {
$schema: https://vega.github.io/schema/vega-lite/v5.json
data: {
url: {
%context%: true
%timefield%: timestamp

  index: terrena_1
  body: {
    size: 0
    aggs: {
      month_date_histogram_of_timestamp: {
        date_histogram: {field: "timestamp", calendar_interval: "month"}
        aggs: {
          sum_of_Jobsize: {
            sum: {field: "Job_Size_(GB)"}
          }
          serial_diff_of_sum_of_Jobsize: {
            serial_diff: {buckets_path: "sum_of_Jobsize", lag: 1}
          }
        }
      }
    }
  }
}

format: {
  property: aggregations.month_date_histogram_of_timestamp.buckets
}

}
transform: [
{
window: [
{
op: lead
field: key_as_string
offset: 1
as: next_lead
}
]
}

{
  calculate: "datum.sum_of_Jobsize.value",
  as: "sum"
}

{
  calculate: "datum.key_as_string",
  as: "lead"
}

{
  calculate: datum.serial_diff_of_sum_of_Jobsize != null ? datum.sum - datum.serial_diff_of_sum_of_Jobsize.value : 0
  as: previous_sum
}
{
  calculate: datum.serial_diff_of_sum_of_Jobsize != null ? datum.serial_diff_of_sum_of_Jobsize.value :datum.sum
  as: amount
}
{
  calculate: (datum.serial_diff_of_sum_of_Jobsize != null && datum.amount (GB > 0 ? '+' : '') + datum.amount
  as: text_amount
}
{
  calculate: (datum.sum + datum.previous_sum) / 2
  as: center
}
{
  calculate: datum.sum < datum.previous_sum ? datum.sum : ''
  as: sum_dec
}
{
  calculate: datum.sum > datum.previous_sum ? datum.sum : ''
  as: sum_inc
}

]
encoding: {
x: {
field: key_as_string
type: ordinal
sort: null
axis: {labelAngle: 0, title: "Months"}
}
}
layer: [
{
mark: {type: "bar", size: 45}
encoding: {
y: {field: "previous_sum", type: "quantitative", title: "Job Size"}
y2: {field: "sum"}
color: {
condition: [
{
test: datum.serial_diff_of_sum_of_Jobsize == null
value: "#f7e0b6"
}
{test: "datum.sum < datum.previous_sum", value: "#f78a64"}
]
value: "#93c4aa"
}
}
}
{
mark: {
type: rule
opacity: 1
color: "#404040"
strokeWidth: 2
xOffset: -22.5
x2Offset: 22.5
}
encoding: {

    x2: {field: "next_lead"}
    y: {field: "sum", type: "quantitative"}
  }
}
{
  mark: {type: "text", dy: -4, baseline: "bottom"}
  encoding: {
    y: {field: "sum_inc", type: "quantitative"}
    text: {field: "sum_inc", type: "nominal"}
  }
}
{
  mark: {type: "text", dy: 4, baseline: "top"}
  encoding: {
    y: {field: "sum_dec", type: "quantitative"}
    text: {field: "sum_dec", type: "nominal"}
  }
}
{
  mark: {type: "text", fontWeight: "bold", baseline: "middle"}
  encoding: {
    y: {field: "center", type: "quantitative"}
    text: {field: "text_amount", type: "nominal"}
    color: {
      condition: [
        {
          test: datum.serial_diff_of_sum_of_Jobsize == null
          value: "#725a30"
        }
      ]
      value: white
    }
  }
}

]
config: {
text: {fontWeight: "bold", color: "#404040"}
}
}

`

Stratoula_Kalafateli · May 31, 2021, 11:04am

Hey @Krunal

I haven't checked it all but it seems that this

{
      calculate: (datum.serial_diff_of_sum_of_Jobsize != null && datum.amount (GB > 0 ? '+' : '') + datum.amount
      as: text_amount
    }

has a parenthesis that doesn't close.

Krunal · May 31, 2021, 1:11pm

@Stratoula_Kalafateli Thanks

Krunal · May 31, 2021, 1:49pm

As i have done some modification then this error came, i put field value in the "Job Size (GB)" bcoz of EsError

Stratoula_Kalafateli · May 31, 2021, 2:05pm

I think that the error is still in the same calculate function that I posted above. To be honest I don't understand what you are trying to calculate there.

Krunal · May 31, 2021, 2:23pm

I would like to see my data in the waterfall chart .

Stratoula_Kalafateli · May 31, 2021, 2:29pm

Nice! I mean that it is not clear to me what you are trying to calculate here

{
      calculate: (datum.serial_diff_of_sum_of_Jobsize != null && datum.amount (GB > 0 ? '+' : '') + datum.amount
      as: text_amount
    }

I think that the bug.is in this function

Krunal · May 31, 2021, 2:57pm

I solved that error
It was my mistake .....calculate:( datum.serial_diff_of_sum_of_Jobsize != null && datum.amount > 0 ? '+' : '') + datum.amount as: text_amount
Now its perfect but still there is some problem

Krunal · June 1, 2021, 8:58am

Hello Expert,
I have stucked again and not getting output....

Stratoula_Kalafateli · June 1, 2021, 10:44am

To be honest it works for me. This is my working example by using a sample dataset:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "title": "Event counts from all indexes",
  "data": {
    "url": {
      "%context%": true,
      "%timefield%": "@timestamp",
      "index": "kibana_sample_data_logs",
      "body": {
          "aggs": {
      "month_date_histogram_of_timestamp": {
        "date_histogram": {
          "field": "timestamp", 
          "calendar_interval": "month"
        },
        "aggs": {
          "sum_of_Jobsize": {
            "sum": {"field": "bytes"}
          },
          "serial_diff_of_sum_of_Jobsize": {
            "serial_diff": {"buckets_path": "sum_of_Jobsize", "lag": 1}
          }
        }
      }
    },
    "size": 0
      }
    },
    "format": {"property": "aggregations.month_date_histogram_of_timestamp.buckets"}
  },
    transform: [
    {
      window: [
        {
          op: lead
          field: key_as_string
          offset: 1
          as: next_lead
        }
      ]
    }
    {
  calculate: "datum.sum_of_Jobsize.value",
  as: "sum"
}

{
  calculate: "datum.key_as_string",
  as: "lead"
}

{
  calculate: datum.serial_diff_of_sum_of_Jobsize != null ? datum.sum - datum.serial_diff_of_sum_of_Jobsize.value : 0
  as: previous_sum
}
{
  calculate: datum.serial_diff_of_sum_of_Jobsize != null ? datum.serial_diff_of_sum_of_Jobsize.value :datum.sum
  as: amount
}
{
  calculate: ( datum.serial_diff_of_sum_of_Jobsize != null && datum.amount > 0 ? '+' : '') + datum.amount 
  as: text_amount
}
{
  calculate: (datum.sum + datum.previous_sum) / 2
  as: center
}
{
  calculate: datum.sum < datum.previous_sum ? datum.sum : ''
  as: sum_dec
}
{
  calculate: datum.sum > datum.previous_sum ? datum.sum : ''
  as: sum_inc
}
  ],
  "mark": "line",
  encoding: {
    x: {
      field: key_as_string
      type: ordinal
      sort: null
      axis: {
        labelAngle: 0
        title: Months
      }
    }
  }
    layer: [
    {
      mark: {
        type: bar
        size: 45
      }
      encoding: {
        y: {
          field: previous_sum
          type: quantitative
          title: Job Size
        }
        y2: {
          field: sum
        }
        color: {
          condition: [
            {
              test: datum.serial_diff_of_sum_of_Jobsize == null
              value: "#f7e0b6"
            }
            {
              test: datum.sum < datum.previous_sum
              value: "#f78a64"
            }
          ]
          value: "#93c4aa"
        }
      }
    }
    {
      mark: {
        type: rule
        opacity: 1
        color: "#404040"
        strokeWidth: 2
        xOffset: -22.5
        x2Offset: 22.5
      }
      encoding: {
        x2: {
          field: next_lead
        }
        y: {
          field: sum
          type: quantitative
        }
      }
    }
    {
      mark: {
        type: text
        dy: -4
        baseline: bottom
      }
      encoding: {
        y: {
          field: sum_inc
          type: quantitative
        }
        text: {
          field: sum_inc
          type: nominal
        }
      }
    }
    {
      mark: {
        type: text
        dy: 4
        baseline: top
      }
      encoding: {
        y: {
          field: sum_dec
          type: quantitative
        }
        text: {
          field: sum_dec
          type: nominal
        }
      }
    }
    {
      mark: {
        type: text
        fontWeight: bold
        baseline: middle
      }
      encoding: {
        y: {
          field: center
          type: quantitative
        }
        text: {
          field: text_amount
          type: nominal
        }
        color: {
          condition: [
            {
              test: datum.serial_diff_of_sum_of_Jobsize == null
              value: "#725a30"
            }
          ]
          value: white
        }
      }
    }
  ]
    config: {
    text: {
      fontWeight: bold
      color: "#404040"
    }
  }
}

and the result is

Krunal · June 2, 2021, 8:13am

Thanks @Stratoula_Kalafateli
I copied your code and pasted in vega, I have changed the field name and index name, then also i am not getting output.

Is there any problem of version of kibana ? Bcoz I am using 7.13

Stratoula_Kalafateli · June 2, 2021, 8:22am

No, because my code also works for 7.13. Have you checked your timefilter?

Maybe you don't have data for the selected dates.

Krunal · June 2, 2021, 8:32am

Stratoula_Kalafateli · June 2, 2021, 8:35am

What type is the Job_Size_(GB) field? Have you run the es query? Does it return data?

Krunal · June 2, 2021, 8:42am

Job Size (GB) is float , yes es query working properly

Stratoula_Kalafateli · June 2, 2021, 8:48am

But here I see that the field is Job Size (GB) and not Job_Size_(GB) or do you use custom labels?

Krunal · June 2, 2021, 8:56am

I have not used Job_Size_(GB) anywhere.
{ "$schema": "https://vega.github.io/schema/vega-lite/v5.json", "title": "Daily Difference of Job Size ", "data": { "url": { "%context%": true, "%timefield%": "@timestamp", "index": "terrena_1", "body": { "aggs": { "day_date_histogram_of_timestamp": { "date_histogram": { "field": "timestamp", "calendar_interval": "day" }, "aggs": { "sum_of_Jobsize": { "sum": {"field": "Job Size (GB)"} }, "serial_diff_of_sum_of_Jobsize": { "serial_diff": {"buckets_path": "sum_of_Jobsize", "lag": 1} } } } },

Stratoula_Kalafateli · June 2, 2021, 9:04am

In this screenshot the field is with empty spaces

Krunal · June 2, 2021, 9:07am

yes . but i tried both Job_Size_(GB) and Job Size (GB) also but i think it isnot able to fetch data ..... why I didn't understand.

Stratoula_Kalafateli · June 2, 2021, 9:11am

I am afraid that the problem is that your field name has spaces. This is something that we do not recommend. Maybe you can try one of the suggestions here Discover field names with spaces