Using Timelion with alphanumeric field names

[ccullen (Caleb Cullen)]

Lately I've been stuffing mail-related logs into ES, and among those are some performance statistics produced by AMaViS -- particularly, data related to how long each step of the virus checking takes. I use inline Ruby to munge the message into a bunch of key-value pairs under the general heading 'timing', so I end up with [timing][parse] and [timing][extract_message_metadata] etc. and also some like [timing][tests_pri__1000] (because my Ruby tr's "- ." into "_"; it appears in the source log message as tests_pri_-1000) ...

Now of course, in Timelion, the query looks like:

.es(index=logstash-*, q=action:timing, metric=avg:timing.parse)

The above works just fine. There are 20-odd datapoints, and all the ones which are only letters and underscores work fine. But when I try to graph the data under timing.tests_pri__1000 it silently fails to gather any data for that particular query. All the others continue to work, but I would also like to be able to graph the data held in the alphanumerically named fields.

Currently, I am at the point of remapping [0-9] onto some arbitrary set of letters, {'0' => 'O', '1' => 'I', ...} etc, but I thought I should raise this issue and ask whether I'm simply doing something wrong, or do I need to escape my numbers, or is this a known shortcoming of Timelion?

[rashmi (kulkarni)]

Sorry , I have not got much idea on this...@timroes - any idea?

[timroes]

Hey Caleb,

could you elaborate a bit more on "when I try to graph the data under timing.tests_pri__1000"?

What does your timelion expression for that looks like? Is the field used in the query (q) parameter or in the metric?

One suggestion would be: could you try putting that whole parameter into quotes (e.g. like q="action:timing") and check if that solves the issue?

Cheers,
Tim

[ccullen (Caleb Cullen)]

Sometimes, when I work with these tools, I feel like I'm losing my mind. I was using a very simple scenario previously when I tried an expression like:

.es(index=logstash-*, q=action:timing, metric=avg:timing.tests_pri__1000)

and got seemingly no result.

But once again, re-testing it now, I find that in fact, that expression works fine. It obtains data from the field name which contains numbers, and it graphs that data. Last week, it seemed as though it silently ignored field names which contained numbers.

It is possible that I simply had other data in the same chart with values up to two orders of magnitude greater in value, which caused the graph to be a flat line.

Thanks for your response to my concern, and for your patience.

[timroes]

Hi Caleb,

glad it works now for you. A small tip (for the case you are not aware of anyway already):

If you want to preclude, that you have values on a complete different scale and thus not seeing the line, add a .yaxis(2) to the expression you are testing. It will place it on the 2nd yaxis and thus make it independent of the data range of the first yaxis. (Any other number works of course, too.)

Cheers,
Tim