Time series of various processing numbers

Elastic Stack Elasticsearch
1

I have several logs that have periodic messages with performance data. I want to graph the metrics over time. Things like number of users logged in, virtual memory used, pages in and out, etc.

Each record/message has a date and time stamp, which I have been able to set as the timefield. Then I have a collection of fields. Most are integer values, some are float and the rest are string.

I have not been able to figure out what type of aggregation I need to set on the values.

2

hey,

as you have not mentioned what result you are expecting it is hard to help in this case. Maybe just take the time to explain the data format of a single document, and the expectation of a result for one field, like number_of_users_logged_in.

What result to you expect when there are more documents in a single bucket? An avg? max/min? etc...

without those expectations it will be super hard to help.

--Alex

3

Alex,

Thank you for your reply!

A record from this log file has a common header, including a timestamp that I capture, and then different content for the process producing the record. I am currently working with a record type that looks like the following.

SYSINFO:Application=GROUP; PID=424; NumUsers=2; CPU(Cores)=10; NumTasks=1; NumErrors=0; PhysicalMem=34359201792; UsedPhysicalMem=8577638400; ProcUsedPhysicalMem=6175686656; VirtualMem=140737488224256; UsedVirtualMem=8384761856; ProcUsedVirtualMem=6286733312; UsedCPU=6.12255; ProcUsedCPU=5.16628; NumCubesInRAM=280; NumDataRecordsInRAM=4824935; NumRecordsInLargestCube=172758

I have a grok filter that put all of those fields in a group based on the record type. I use grok so that I can type the fields integer instead of doing a separate mutate{}. I end up with the following.

Data2.UsedVirtualMem = 8384761856
Data2.NumUsers = 2
etc.

What I want to get is a line chart with date and time on the X axis and then the various fields on the Y. Give the wide range, I would use one side for the large memory number and the side for the smaller values like users, tasks and such.

I don't want to aggregate the values, I want to graph them.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.