So sorry for the lack of response @Nikhil_Utane.
I believe the answer is that it's not possible yet using the Time Series Visual Builder. See the response to my post in [TSVB] Metric count shows 0 · Issue #13370 · elastic/kibana · GitHub
That's a limitation of pipeline aggs, you can't use a sibling pipeline agg in a bucket_script.
Which eliminates what I was trying to do with the Visual Builder metric.
In addition, you can't use scripted fields with the Visual Builder - Scripted fields can not be used in Visual Builder · Issue #11999 · elastic/kibana · GitHub
so I don't think you can solve the issue that way either. So unfortunately, I don't think this is possible at the moment.