Time series of various processing numbers

I have several logs that have periodic messages with performance data. I want to graph the metrics over time. Things like number of users logged in, virtual memory used, pages in and out, etc.

Each record/message has a date and time stamp, which I have been able to set as the timefield. Then I have a collection of fields. Most are integer values, some are float and the rest are string.

I have not been able to figure out what type of aggregation I need to set on the values.


as you have not mentioned what result you are expecting it is hard to help in this case. Maybe just take the time to explain the data format of a single document, and the expectation of a result for one field, like number_of_users_logged_in.

What result to you expect when there are more documents in a single bucket? An avg? max/min? etc...

without those expectations it will be super hard to help.



Thank you for your reply!

A record from this log file has a common header, including a timestamp that I capture, and then different content for the process producing the record. I am currently working with a record type that looks like the following.

SYSINFO:Application=GROUP; PID=424; NumUsers=2; CPU(Cores)=10; NumTasks=1; NumErrors=0; PhysicalMem=34359201792; UsedPhysicalMem=8577638400; ProcUsedPhysicalMem=6175686656; VirtualMem=140737488224256; UsedVirtualMem=8384761856; ProcUsedVirtualMem=6286733312; UsedCPU=6.12255; ProcUsedCPU=5.16628; NumCubesInRAM=280; NumDataRecordsInRAM=4824935; NumRecordsInLargestCube=172758

I have a grok filter that put all of those fields in a group based on the record type. I use grok so that I can type the fields integer instead of doing a separate mutate{}. I end up with the following.

Data2.UsedVirtualMem = 8384761856
Data2.NumUsers = 2

What I want to get is a line chart with date and time on the X axis and then the various fields on the Y. Give the wide range, I would use one side for the large memory number and the side for the smaller values like users, tasks and such.

I don't want to aggregate the values, I want to graph them.