Skip to content

fix: Tempo metricsGenerator + Alloy logpods 수집 개선#54

Merged
doup2001 merged 3 commits intomainfrom
fix/#49
Apr 11, 2026
Merged

fix: Tempo metricsGenerator + Alloy logpods 수집 개선#54
doup2001 merged 3 commits intomainfrom
fix/#49

Conversation

@doup2001
Copy link
Copy Markdown
Member

@doup2001 doup2001 commented Apr 11, 2026

📌 작업한 내용

  • Tempo metricsGenerator 활성화: 트레이스 데이터에서 메트릭 자동 생성
  • Alloy logpods 기반 수집: Kubernetes Pod 로그 수집 방식 최적화

🔍 참고 사항

  • Tempo에서 서비스별 Span 메트릭(RPM, Latency, Error Rate) 자동 생성
  • Alloy logpods discovery로 K8s 환경의 모든 Pod 로그 안정적 수집
  • Grafana 대시보드에서 실시간 트레이스 메트릭 확인 가능
  • PinHouse_CLOUD 관찰성 스택 완전 구축 (로그→메트릭→트레이스)

🖼️ 스크린샷

🔗 관련 이슈

#49 (Grafana Loki/Tempo 설정 개선)

✅ 체크리스트

  • 로컬에서 빌드 및 테스트 완료
  • Grafana에서 Tempo 메트릭 확인
  • Alloy 로그 수집 정상화 확인
  • 코드 리뷰 반영 완료

Summary by CodeRabbit

릴리스 노트

  • 새로운 기능
    • Pod 로그 수집을 Kubernetes 커스텀 리소스로 조건부 배포 가능 (podLogs 활성화)
    • 분산 추적에서 메트릭 생성기(metricsGenerator) 활성화 및 단일 복제본·자원 설정
  • 변경
    • 로그 파이프라인 재구성: Pod 로그 수집 경로 변경 및 기존 relabel 단계 제거
    • Alloy 관련 배포 설정에 노드 정보(NODE_NAME) 환경변수 주입 및 CRD 생성 옵션 추가

@doup2001 doup2001 self-assigned this Apr 11, 2026
@doup2001 doup2001 added the fix 기능 수정 및 버그 픽스 label Apr 11, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 11, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c8735159-1ef1-4fcb-8c41-6d15a5e2e0ca

📥 Commits

Reviewing files that changed from the base of the PR and between 5f88ec9 and 3134f30.

📒 Files selected for processing (2)
  • k8s-helm/releases/monitoring-alloy/values.yaml
  • k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml

Walkthrough

Alloy에 PodLogs CR 템플릿을 추가하고 Alloy 로그 파이프라인을 relabel → process로 재배선했으며, Tempo에서 메트릭 생성기(metricsGenerator)를 활성화·구성하는 Helm 값 변경이 포함됩니다.

Changes

Cohort / File(s) Summary
Alloy PodLogs 템플릿
k8s-helm/releases/monitoring-alloy/templates/podlogs.yaml
.Values.podLogs.enabled이 true일 때 Grafana Alloy PodLogs CR을 생성. 메타데이터(이름/네임스페이스/라벨)와 Kubernetes 메타데이터를 표준 레이블(namespace, pod, container, node)로 매핑하는 relabel 규칙 및 복합 service 추출 로직 포함.
Alloy Helm 값 변경
k8s-helm/releases/monitoring-alloy/values.yaml
alloy.crds.create: true, alloy.alloy.extraEnvNODE_NAME 주입 추가. podLogs.enabled: true 추가. Loki 파이프라인을 loki.source.podlogsloki.process.pod_logs.receiver로 재배선하고 기존 loki.relabel "pod_logs" 단계를 제거.
Tempo 메트릭 생성기 값 변경
k8s-helm/releases/monitoring-tempo/values.yaml, k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml
tempo.metricsGenerator.enabled를 true로 변경하고 replicas: 1, CPU/메모리 리소스 요청·제한 추가. prod-gitops에 RED 메트릭 출력용 config.storage.remote_write 대상과 overrides.defaults.metrics_generator.processors.span-metrics 설정 추가.

Sequence Diagram(s)

sequenceDiagram
    participant K8s as Kubernetes Cluster
    participant PodLogs as PodLogs CR
    participant Alloy as Alloy Agent
    participant Loki as Loki

    K8s->>PodLogs: Pod 메타데이터 제공 (namespace, pod, container, node)
    PodLogs->>PodLogs: Relabel 규칙 적용 (service, service_namespace, job 등 산출)
    PodLogs->>Alloy: 레이블이 풍부한 로그 전송
    Alloy->>Alloy: loki.process.pod_logs로 처리
    Alloy->>Loki: 로그 전송 (리모트/수신기)
    Loki->>Loki: 메타데이터와 함께 로그 저장
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 새벽에 깡충, 템플릿이 싹 트네
라벨을 엮어 서비스 이름을 찾아내고
파이프라인은 길을 바꿔 흘러가네
메트릭은 깨어나 리듬을 만들고
모니터링 밭에 당근빛 알림이 반짝! 🎉

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed PR 제목은 Tempo metricsGenerator 활성화와 Alloy logpods 수집 개선이라는 주요 변경사항을 정확하게 반영하고 있습니다.
Description check ✅ Passed PR 설명은 템플릿의 모든 주요 섹션(작업한 내용, 참고 사항, 관련 이슈, 체크리스트)을 완성하였으나 스크린샷 섹션은 비어있습니다.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/#49

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@k8s-helm/releases/monitoring-alloy/templates/podlogs.yaml`:
- Line 1: The template currently only checks .Values.podLogs.enabled which will
fail if the cluster lacks the monitoring.grafana.com/v1alpha2 CRD; update the
podlogs.yaml template to guard resource creation by checking the API presence
(use .Capabilities.APIVersions.Has "monitoring.grafana.com/v1alpha2") in the
same conditional that references .Values.podLogs.enabled, or alternatively
document this CRD as a required precondition in Chart.yaml and README; ensure
you reference .Values.podLogs.enabled and .Capabilities.APIVersions.Has in the
same if/else flow so the template never renders CRD objects on clusters that do
not support them.

In `@k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml`:
- Around line 85-100: The metrics-generator is enabled but you haven't declared
the processor needed to emit RED metrics; add the span-metrics processor under
overrides.defaults.metrics_generator.processors so Tempo actually generates
RPM/latency/error series (i.e., ensure metricsGenerator.enabled: true remains
and configure overrides.defaults.metrics_generator.processors to include
"span-metrics" with any required settings for Tempo 1.61.2 and your remote_write
target).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b8f3365f-aae1-43bb-98d6-03b8b91f153e

📥 Commits

Reviewing files that changed from the base of the PR and between 374fe0a and 5f88ec9.

📒 Files selected for processing (4)
  • k8s-helm/releases/monitoring-alloy/templates/podlogs.yaml
  • k8s-helm/releases/monitoring-alloy/values.yaml
  • k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml
  • k8s-helm/releases/monitoring-tempo/values.yaml

@@ -0,0 +1,87 @@
{{- if .Values.podLogs.enabled }}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

find . -name "podlogs.yaml" -type f

Repository: PinHouse/PinHouse_CLOUD

Length of output: 126


🏁 Script executed:

fd "podlogs.yaml" -type f

Repository: PinHouse/PinHouse_CLOUD

Length of output: 237


🏁 Script executed:

git ls-files | grep -i podlog

Repository: PinHouse/PinHouse_CLOUD

Length of output: 124


🏁 Script executed:

cat -n ./k8s-helm/releases/monitoring-alloy/templates/podlogs.yaml

Repository: PinHouse/PinHouse_CLOUD

Length of output: 2826


🏁 Script executed:

# Check for CRD guard patterns in the entire templates directory
rg "Capabilities.APIVersions" ./k8s-helm/releases/monitoring-alloy/templates/ -A 2 -B 2

Repository: PinHouse/PinHouse_CLOUD

Length of output: 49


🏁 Script executed:

# Check other template files for comparison
ls -la ./k8s-helm/releases/monitoring-alloy/templates/

Repository: PinHouse/PinHouse_CLOUD

Length of output: 239


🏁 Script executed:

# Check if there's any documentation about CRD prerequisites
rg "PodLogs\|monitoring.grafana.com" ./k8s-helm/ --type md -i -A 3 -B 3

Repository: PinHouse/PinHouse_CLOUD

Length of output: 49


🏁 Script executed:

# Check values.yaml for podLogs section
find ./k8s-helm -name "values.yaml" -type f | xargs grep -l "podLogs" 2>/dev/null

Repository: PinHouse/PinHouse_CLOUD

Length of output: 115


🏁 Script executed:

cat -n ./k8s-helm/releases/monitoring-alloy/values.yaml | grep -A 10 -B 2 "podLogs"

Repository: PinHouse/PinHouse_CLOUD

Length of output: 226


🏁 Script executed:

# Check if there's a README or CHANGELOG mentioning CRD requirements
find ./k8s-helm -name "README*" -o -name "CHANGELOG*" -o -name "*.md" | head -20

Repository: PinHouse/PinHouse_CLOUD

Length of output: 49


🏁 Script executed:

# Check the Chart.yaml for any documentation references
cat ./k8s-helm/releases/monitoring-alloy/Chart.yaml

Repository: PinHouse/PinHouse_CLOUD

Length of output: 364


이 템플릿을 배포하려면 클러스터에 monitoring.grafana.com/v1alpha2 CRD가 필수입니다.

현재 1번 줄에서 .Values.podLogs.enabled 값만으로 조건을 판단하고 있습니다. 대상 클러스터에 PodLogs CRD가 설치되어 있지 않으면 Helm 설치/업그레이드가 실패합니다. .Capabilities.APIVersions.Has 같은 가드를 추가하거나, Chart.yaml이나 README에서 필수 전제 조건으로 명시해야 합니다.

🧰 Tools
🪛 YAMLlint (1.38.0)

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@k8s-helm/releases/monitoring-alloy/templates/podlogs.yaml` at line 1, The
template currently only checks .Values.podLogs.enabled which will fail if the
cluster lacks the monitoring.grafana.com/v1alpha2 CRD; update the podlogs.yaml
template to guard resource creation by checking the API presence (use
.Capabilities.APIVersions.Has "monitoring.grafana.com/v1alpha2") in the same
conditional that references .Values.podLogs.enabled, or alternatively document
this CRD as a required precondition in Chart.yaml and README; ensure you
reference .Values.podLogs.enabled and .Capabilities.APIVersions.Has in the same
if/else flow so the template never renders CRD objects on clusters that do not
support them.

Comment thread k8s-helm/releases/monitoring-tempo/values-prod-gitops.yaml
@doup2001 doup2001 merged commit 5844455 into main Apr 11, 2026
1 check passed
@doup2001 doup2001 deleted the fix/#49 branch April 11, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix 기능 수정 및 버그 픽스

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant