[BUG?] Escaping periods in vega-lite calculate transform

[tadgh]

Using the percentile aggregate, elasticsearch returns results wherein the key contains a period. Like so:

 "score_outlier": {
            "values": {
              "1.0": 1,
              "5.0": 1,
              "25.0": 2,
              "50.0": 3,
              "75.0": 3,
              "95.0": 5,
              "99.0": 5
            }
          }

When visualizing, vega lite indicates that you should escape any fields containing periods using a double backslash: \\.

This works totally fine in the encodings, like here:

  {
      "mark": {"type": "bar", "style": "box"},
      "encoding": {
        "y": {"field": "score_outlier.values.25\\.0", "type": "quantitative"},
        "y2": {"field": "score_outlier.values.75\\.0", "type": "quantitative"},
        "x": {"field": "key", "type": "nominal"},
        "color": {"value": "steelblue"},
        "size": {"value": 11}
      }
    }

But if I do the same thing during a calculate transform, I get a Parse Error from Vega. Here is an example which fails:

   "transform": [
     {
       "calculate": "datum.score_outlier.values.75\\.0 - datum.score_outlier.values.25\\.0",
       "as": "iqr"
     }
   ]

Which returns:

Expression parse error: "datum.score_outlier.values.75\\.0 - datum.score_outlier.values.25\\.0"

I've tried not escaping, double-escaping, using square bracket indexing, but nothing seems to work. Am I missing something? The moment I switch this calculation to use a field that doesn't contain a period, all is well.

Thanks for any insights

[rashmi (kulkarni)]

@nyuriks - any thoughts here?

[tadgh]

To quick-fix the issue, I tried just adding a bucket script aggregation, which turns out, suffers the same issue I am seeing here I think.

I actually think the issue may be higher up, in Elasticsearch: Bucket Script Aggregating on fields that contain a period . To narrow this down, I made a small example query. This may not be VEGA-related after all, but just ES related.

[nyuriks (Yuri Astrakhan)]

@tadgh for the Vega part, try using index access instead, e.g. datum.score_outlier.values['75.0']

[tadgh]

Thanks, changing my transform to:

  "transform": [
    {
      "calculate": "datum.score_outlier.values['75.0'] - datum.score_outlier.values['25.0']"
      "as": "iqr"
    },

Did the trick. That said, seems a bit off that its fine in the encoding, but breaks in the calculation. Slightly unintuitive to have to nest quotes in one spot, but able to escape in another.

[nyuriks (Yuri Astrakhan)]

@tadgh while not ideal, the reason it works in the field is because field is not an expression, it is a path to a value inside the current datum. So if datum is {a: {b: [10.5]: 'something'}}, getting to something would be a.b.10\\.5. On the other hand, signal and expr values are real expressions - they get evaluated similar to how javascript evaluates things. JS would throw an error with datum.a.b.10\\.5, and so does Vega.

[tadgh]

Understood, thanks for the clarification