Add TimerHeapMetrics

[iRevive]

A follow-up for #3317 (comment).

I'm unfamiliar with the TimerHeap internals, so perhaps the direction needs to be corrected.

[armanbilge]

thanks so much for starting this follow-up :)

[armanbilge]

We should figure out what this Long represents. Currently it is an absolute time in nano seconds relative to the (arbitrary) nanoTime offset. I am not sure how useful that is.

• Would it make sense to convert to a unix epoch time? I think we'd have to give up some precision.
• Or, we could return the number of nanoseconds until the next timer fires — so a relative time to the present, not absolute.

[iRevive]

the number of nanoseconds until the next timer fires — so a relative time to the present, not absolute.

This option looks more versatile and suitable.

[armanbilge]

Once we decide on the API I can help with the implementation. I assume that this method may be called from non-owning threads, in which case we need to write the implementation defensively against race conditions.

I also think we should just return the time at the root of the heap, even if it's deleted/canceled. Returning None is confusing when there are more active timers besides the root. Besides, the timer was active at some point, deletion/cancelation is a race condition against metrics retrieval, and it's still a lower bound on the actual next timer due. So I think it's still relevant statistically speaking.

[iRevive]

From the OpenTelemetry perspective, I can think of the following metrics:

• timerheap.scheduled.count - total number of timers that have been scheduled (enqueued)
• timerheap.executed.count - total number of executed timers
• timerheap.cancelled.count - total number of canceled timers - canceledCounter.get() or removedCanceledCounter
• timerheap.queue.size - current number of pending timers - size
• timerheap.next.execution.time - the due of the next timer - nextTimerDue

We may need to introduce new counters to keep track of scheduled and executed timers.