Canvas Blog Stream

(Rashid Khan) #1

We spent some time today talking about how to do multi "series" transformation on the data table, which led me to make a change to mapColumn. The mapColumn function now receives each row as a single row datatable. I find it more pleasant to use, and if you need to get a specific column value you can just use getCell myColumn

(Rashid Khan) #2


This new time filter is super excellent. It is simpler than the core time picker by design.

Canvas workpads are tailored experiences and we want you to craft each element to their exact needs. For this reason we plan to make the time picker customizable. The quick ranges won't have to be static, they could be set to a list defined by you.

The large view is visible here, but we can make this collapsible into a button that just displays the currently selected time.

(Rashid Khan) #3

I added simple support for Timelion.

Because timelion requires a time range do to its magic, I've set it up so that it takes a to and from,
but can also make use of a timefilter element. If you have a timefilter, it will use that, but the local to and from with override it.

(Rashid Khan) #4

You can get soooo tacky with this. Whatever, it's your ball game, play it how you want. We really need to get a library of styles going.

(Rashid Khan) #5


Sort of a silly example of how sub-expressions work, but this sizes every point randomly. The numeric value of point= is generated by math

esdocs index="usagov*" | pointseries y="bytes" x="@timestamp" color="geo.country_code" | plot defaultStyle={seriesStyle points={math "random() * 20"}}

(Rashid Khan) #6

We're planning to create a library of example Canvas workpads. Here's an example...of an example (ed note: Oooh meta). This shows you how to create a custom color palette for use with pie, plot, grid and anything you create that needs a palette

To try it out...

  1. Download the example file
  2. Open the workpad loader inside of Canvas by clicking on "Workpads"
  3. Drag the file you downloaded onto the loader

(Rashid Khan) #7

We just cut a new release (0.1.1653) that includes update notifications. Keep an eye out for the goat and information on installing new builds.
We're also working to include the change information in that update notification, but that isn't there just yet.

Want to install the new build? The Installation Instructions have been updated

(Rashid Khan) #8

No point in rolling this into a release just yet, but I started working on support for Elasticsearch SQL so we can hit the ground running the day it's released

(Rashid Khan) #9

Faster page loads!

Just pushed out 0.1.1676 which makes a big change in how we manage the page stack. Previously we would render each page as it loaded. This was pretty much fine, but it meant that every element's render function had to be really fast, otherwise there would be a blank spot on the page for a moment.

I was hacking on a fork of Joe's timeline plugin and it was just a tiny bit too slow. There really wasn't much I could do about it, the upstream library wasn't fast. Now Canvas makes up for that!

(Rashid Khan) #10

Sub-expressions now receive context

This change is actually a change back. When Joe and I were designing the Canvas expression language we decided that sub-expressions would receive the same context as the function they were an argument too. We removed that for a long time because it caused problems when type checking the input context for the subexpressions. Wat?

Ok, let me explain, take a look at:

foo | bar baz={beer}

Up until now, in every released version, bar would receive the output of foo, but beer would receive no context all, aka a null context. With the change I just made beer will now receive the same input as foo.

This didn't work previously because what if beer required a null context? Well presumably foo doesn't output beer, so you would receive an error. A few updates again, quietly, we I introduced the ability for the interpreter to cast from a special wildcard type. Now there's a from on the null types that says "Here's how you make a null from anything". With that change, the interpreter sees that foo outputs whatever, and beer requires null, and it looks at the null type and says "Ok, I don't know how to convert whatever, but this '*' rule says to just run this function and I'll get a null from anything, I'll do that"

This change mostly impacts conditional statements. Now you can do stuff like:

esdocs index="heartbeat*" fields="," sort="@timestamp, desc"
 | mapColumn "isAwesome" fn=${
  if {getCell "" | compare "lt" to=200000} then="Awesome" else="Just OK"
 | mapColumn "isAwesome" fn=${
  if {getCell "" | compare "gt" to=3000000} then="Not Awesome" else={getCell "isAwesome"}

This says: Pull a couple fields from the docs in the heartbeat* index, sorted by @timestamp. Now create a new column called isAwesome and if the field is less than 200000, set isAwesome to "Awesome". Otherwise set it to "Just OK". Then if that field is greater than 3000000 set isAwesome to "Not Awesome", otherwise just set it to whatever it currently is. This all works because the context is being passed deeply down the nested sub-expressions. Neat right?

(Rashid Khan) #11

1736 and the alignment grid

There's actually been a bunch of changes lately, but this was one someone had asked for and was a quick and easy win. You can now hit ALT+G to toggle an alignment grid. Eventually we'll add snapping to the layout engine and this won't be needed anymore, but it works for now.

(Rashid Khan) #12

Image Reveal element


The holidays are well and truly over and we're back!. First up, this awesome new image reveal element. This takes in a number from 0 to 1 and uses it to decide what percentage of an image to show. In the image for this post you can see it slowly moving between 0.3 and 1.0.

It also takes an option background image. To create the filling bottle effect I've set a greyscale version of the image as the background and used a green version of the bottle as the image to reveal. Neat, right?

(Rashid Khan) #13

Image Repeat Element


And the 2nd in our "infographic" focused elements. The image repeat element repeats a supplied image a given number of times up to a specified
maximum. You can also, optionally, supply an image to fill out the number up to the max. In this example we've used a full bottle as the image to
repeat, with an empty bottle filling out the element up to the max. Snazzy.

(Rashid Khan) #14

Look at this nonsense!

You can do some wacky stuff with the custom CSS support in Canvas. We're working on some ElasticON stuff and I stole some assets from our
very talented designers to make this funky isometric map/chart thing. The little bubble chart CSS looks like this:

.canvas__element {
  background-color: #fff;
  transform: skewY(-30deg);
  border-radius: 5px;

(Rashid Khan) #15

Did you know? URL parameters

If you're on this blog you've probably been around the Elastic stack for awhile. Remember Kibana 3? A little known feature of Kibana 3 was
the ability to create "scripted" dashboards. You could use URL parameters to inject strings into the dashboard JSON, but there were all sorts
of limitations. For example, scripted dashboards had to be store on disk. In Canvas you can use URL parameters anywhere in an expression.

Obviously you should be careful here, string concatenation and such, but it's a pretty sweet feature. The urlparam function will grab a
specified URL parameter and return it in you expression, for use by any other function, or just on its own. You can also specify a default
if said URL parameter is missing. Check out the screenshot to see how it works.

(Rashid Khan) #16

Creating new columns: Part 1

Canvas expressions have a few ways of adding a column to a data table. In this post we'll look at the simplest: staticColumn. This function
is simple, effective and fast at add static values and, when combined with a sub-expression, summary columns. In these examples we'll use
the demodata datasource to keep things simple, but of course these all work with your data too.

A simple example that would add a column called hello containing the string "Hello World" would look like this:

demodata | staticColumn hello value="Hello world"

Simple right? The unnamed parameter is the name of the new column to add, and the value is "Hello world". However, we can leverage what we
know about sub-expressions to do so much more. Remember, sub-expression receive the same context as the primary function. For example:

demodata | staticColumn total_sales value={math 'sum(price)'}

In this example, math receives the output of demodata, so we can use it to summarize the price column into a new column called total_sales.
We could now use markdown to construct a simple metric element with a summary of our recent sales history. Again, remember that getCell
will receive the datatable that has been output from staticColumn. Here were accessing the first row since the default for the row
argument to getCell is 0:

 | staticColumn total_sales value={math 'round(sum(price))'}
 | markdown "We've sold **$" {getCell total_sales} "** worth of product.
   Our most recent sale was $" {getCell price}

(Rashid Khan) #17

Hidden gem: Advanced Filter

Sometimes we create hidden or debug functions in Canvas by simply not creating a UI for them. One of those hidden abilities is the "Advanced
Filter" element. The advanced filter really is for advanced users since you need to know how Canvas filters work, but the gist is that it
is an element that allows you to type a Canvas expression to use as a filter anywhere that the filters function is used on a workpad.

To get the Advanced Filters element you'll need to drop into the expression editor and enter:

render as="advanced_filter"

You can then type a filter expression in there. Right now filters are only used by esdocs, escount and timelion. There are currently
2 filter functions available: exactly and timefilter. You can try those out in the expression editor too, just replace filters with
your own filter. For example:

exactly | esdocs index=heartbeat*

Also, don't forget, filters are just regular Canvas expression so you can do sub-expressions. This one will use esdocs and getCell to
lookup the most recent host to report in and filter a subsequent fun of esdocs to only pull back documents with that host:

exactly value={esdocs index=heartbeat* sort="@timestamp,desc"| getCell} | esdocs index=heartbeat*

(Rashid Khan) #18

Fixed that mapping error you might have seen

You may have noticed a console error, something or another about mappings. We fixed that 0.1.1806 by using some really nice Kibana APIs, but this is important, pay attention: You will need to reindex your workpads. Don't worry, this is pretty easy. There are instruction below. The TL;DR is that we'll be backing up your workpads, then deleting them, then reindexing them with new IDs. Nothing will change from your perspective, but if you might want to download the JSON of your workpads before starting this process.

Jump over to Kibana's dev console and do this:

# Backup your canvas workpads in an index called 'canvas-backup-20180206'
POST _reindex
  "source": {
    "index": ".kibana",
    "query": {
      "match": {
        "type": "canvas-workpad"
  "dest": {
    "index": "canvas-backup-20180206"

# Now delete your Canvas workpads in .kibana
POST .kibana/doc/_delete_by_query
  "query": {
    "match": {
      "type": "canvas-workpad"

# And re-index from the backup with the new ID scheme
POST _reindex
  "source": {
    "index": "canvas-backup-20180206"
  "dest": {
    "index": ".kibana"
  "script": {
    "source": "if (ctx._source.type == 'canvas-workpad') {ctx._id = 'canvas-workpad:' + ctx._id}",
    "lang": "painless"

(Rashid Khan) #19

So much faster!

I just released 1813 for Kibana 6.2.1 and it is much, much faster. Specifically when loading and refreshing entire workpads, performance is improved by 75-90%. That means workpads that used to take 20 seconds to load, now take 5 or less. And there's more good news coming on that front: I found a lot more meat on the performance bone, this is really just the low hanging fruit, to mix metaphors. Enjoy!

(Rashid Khan) #20

It's all good

Well, 1813 didn't last long. Not that there was anything wrong with it. 1816 is now available which adds a new "ANY" option to the dropdown filter element. I also fixed the height of the code window so you don't have to resize it all the time.