How to interpret memory stats in filebeat log

[filebeater]

Hello, I am wondering what is the following 3 memstats means?
beat.memstats.gc_next=32389200
beat.memstats.memory_alloc=32504416
beat.memstats.memory_total=51506747192

For example, i would like to know what is maximum total memory consumed by filebeat across all time? can I infer from the above logs that filebeat could consume up to 51G at some point?

[pierhugues (Pier-Hugues Pellerin)]

Hello @filebeater,

Good question to ask, I took the following directly from the go documentation.

beat.memstats.gc_next=32389200

	// NextGC is the target heap size of the next GC cycle.
	//
	// The garbage collector's goal is to keep HeapAlloc ≤ NextGC.
	// At the end of each GC cycle, the target for the next cycle
	// is computed based on the amount of reachable data and the
	// value of GOGC.

beat.memstats.memory_alloc=32504416

	// HeapAlloc is bytes of allocated heap objects.
	//
	// "Allocated" heap objects include all reachable objects, as
	// well as unreachable objects that the garbage collector has
	// not yet freed. Specifically, HeapAlloc increases as heap
	// objects are allocated and decreases as the heap is swept
	// and unreachable objects are freed. Sweeping occurs
	// incrementally between GC cycles, so these two processes
	// occur simultaneously, and as a result HeapAlloc tends to
	// change smoothly (in contrast with the sawtooth that is
	// typical of stop-the-world garbage collectors).

beat.memstats.memory_total=51506747192

   // TotalAlloc is cumulative bytes allocated for heap objects.
   //
   // TotalAlloc increases as heap objects are allocated, but
   // unlike Alloc and HeapAlloc, it does not decrease when
   // objects are freed.

So when you are talking about 51G, i think you are referring to the memory_total, which is a bit misleading, because no you wont use that much memory it only trace how much memory has been allocaled in total.

So the value you should look at is memory_alloc, 32504416 bytes so 30M.

[filebeater]

thanks for the explanation! very helpful!