[FEATURE] Add Prometheus metrics for empty blocks and SR set changes

[warku123]

Background

Currently, java-tron's Prometheus metrics provide comprehensive monitoring for transactions, blocks, P2P connections, and consensus status. However, there are two important operational aspects that lack visibility:

1. Empty Block Detection: There is no metric to track blocks that contain no transactions. Monitoring empty blocks is crucial for identifying potential network issues, misconfigured witness nodes, or degraded transaction throughput.

2. Super Representative (SR) Set Changes: The active witness set changes during maintenance periods when new SRs are elected or removed. Currently, there is no way to track these changes through metrics, making it difficult to monitor consensus layer dynamics and SR rotation events.

Problem Statement & Rationale

• Node operators cannot identify which witness nodes are producing empty blocks, making it difficult to diagnose network health issues
• There is no real-time visibility into SR set changes

What are the use cases?

1. Identify witnesses producing excessive empty blocks
2. Monitor SR election results and consensus transitions

Proposed Solution

Add two new Prometheus counters to track empty blocks and SR set changes:

• tron:block_empty_total - Tracks empty blocks
• tron:sr_set_change_total - Tracks SR set changes (add/remove)

Testing Strategy

Test Scenarios

1. Unit Test: Add unit test about two counters.
2. Manual Test: Build a private chain and test the case include produce empty and SR set change.

Scope of Impact

• Other: Metrics/Monitoring system

Backward Compatibility

Fully backward compatible. Existing metrics remain unchanged. New metrics are opt-in via Prometheus scraping.

Implementation

Do you have ideas regarding the implementation?

The implementation involves:

1. Metric Registration: Add new counter definitions in MetricKeys.java, MetricLabels.java, and MetricsCounter.java

2. BlockChainMetricManager Modifications: In BlockChainMetricManager.java:

   • Add empty block detection logic
   • Add SR set change detection logic

3. Testing: Extend PrometheusApiServiceTest to cover new metrics functionality

Are you willing to implement this feature?

• Yes, I can implement this feature

Estimated Complexity

• Medium (moderate changes)

References

• Prometheus Counter Best Practices

[Federico2014]

Good proposal — the two metrics address real operational blind spots.

One addition to consider: tron:empty_block_ratio as a Gauge showing the ratio of empty blocks in recent N blocks (e.g., last 1000). The counter tells you cumulative totals, but a ratio gauge reveals recent trends — operators can spot degrading witness health when empty blocks start spiking and set alerts like "ratio > 10%". Much more actionable for dashboards.

Implementation would just need a sliding window in applyBlock(). Worth adding?

[Sunny6889]

This TIP addresses important monitoring needs and the overall approach is sound. However, I have a suggestion:

Instead of adding a separate tron:block_empty_total counter, consider using a tron:block_transaction_count histogram to track the distribution of transaction counts per block. This approach offers several advantages:

1. Covers empty block monitoring: The histogram's le="0" bucket directly tracks empty blocks
2. Provides richer insights: Enables analysis of transaction distribution, averages, and percentiles
3. Better trend analysis: Helps identify network load patterns and anomalies
4. One metric, multiple solutions: Satisfies empty block detection while providing network throughput analysis

The current tron:txs counter only records when transactions exist, completely missing empty blocks. A histogram would capture transaction count information for all blocks, giving operators a more comprehensive view of network health.

The SR change monitoring counter approach looks good and should remain unchanged.