apiVersion: jenkins.io/v1 kind: PipelineActivity metadata: annotations: lighthouse.jenkins-x.io/cloneURI: https://github.com/greencapitaltrade/bifrost.git lighthouse.jenkins-x.io/job: release pipeline.jenkins-x.io/traceID: 992ab7196e8610490056b2b71712a78c creationTimestamp: "2026-08-08T13:10:29Z" generation: 8 labels: branch: main build: "198" context: release created-by-lighthouse: "true" external-trigger: "true" lighthouse.jenkins-x.io/baseSHA: 7babab678fbfa1d8a1269bc4047483fb8b3c15ef lighthouse.jenkins-x.io/branch: main lighthouse.jenkins-x.io/buildNum: "1786194629917" lighthouse.jenkins-x.io/context: release lighthouse.jenkins-x.io/id: greencapitaltrade-bifrost-2j7sp lighthouse.jenkins-x.io/job: release lighthouse.jenkins-x.io/lastCommitSHA: 7babab678fbfa1d8a1269bc4047483fb8b3c15ef lighthouse.jenkins-x.io/refs.org: greencapitaltrade lighthouse.jenkins-x.io/refs.repo: bifrost lighthouse.jenkins-x.io/type: postsubmit owner: greencapitaltrade podName: apitaltrade-bifrost-main-release-bqpsm-from-build-pack-pod provider: github repository: bifrost tekton.dev/pipeline: apitaltrade-bifrost-main-release managedFields: - apiVersion: jenkins.io/v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: f:branch: {} f:context: {} f:owner: {} f:provider: {} f:repository: {} f:spec: f:lastCommitMessage: {} f:lastCommitSHA: {} f:releaseNotesURL: {} f:version: {} manager: jx-changelog-0.10.29 operation: Update time: "2026-08-08T13:15:40Z" - apiVersion: jenkins.io/v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:lighthouse.jenkins-x.io/cloneURI: {} f:lighthouse.jenkins-x.io/job: {} f:pipeline.jenkins-x.io/traceID: {} f:labels: .: {} f:build: {} f:created-by-lighthouse: {} f:external-trigger: {} f:lighthouse.jenkins-x.io/baseSHA: {} f:lighthouse.jenkins-x.io/branch: {} f:lighthouse.jenkins-x.io/buildNum: {} f:lighthouse.jenkins-x.io/context: {} f:lighthouse.jenkins-x.io/id: {} f:lighthouse.jenkins-x.io/job: {} f:lighthouse.jenkins-x.io/lastCommitSHA: {} f:lighthouse.jenkins-x.io/refs.org: {} f:lighthouse.jenkins-x.io/refs.repo: {} f:lighthouse.jenkins-x.io/type: {} f:podName: {} f:tekton.dev/pipeline: {} f:spec: .: {} f:baseSHA: {} f:batchPipelineActivity: {} f:build: {} f:buildLogsUrl: {} f:completedTimestamp: {} f:context: {} f:gitBranch: {} f:gitOwner: {} f:gitRepository: {} f:gitUrl: {} f:message: {} f:pipeline: {} f:startedTimestamp: {} f:status: {} f:steps: {} manager: jx-build-controller operation: Update time: "2026-08-08T13:17:47Z" name: greencapitaltrade-bifrost-main-198 namespace: jx resourceVersion: "253025056" uid: e343e8e9-a6b3-4a1e-8fbb-075ed97d1c62 spec: baseSHA: 7babab678fbfa1d8a1269bc4047483fb8b3c15ef batchPipelineActivity: {} build: "198" buildLogsUrl: s3://logs-gct-prod-20260425045301534100000007/jenkins-x/logs/greencapitaltrade/bifrost/main/198.log completedTimestamp: "2026-08-08T13:17:40Z" context: release gitBranch: main gitOwner: greencapitaltrade gitRepository: bifrost gitUrl: https://github.com/greencapitaltrade/bifrost.git lastCommitMessage: | fix(cron): survive the startup race, and notice when crons stop On 2026-08-07 every scheduled job in production stopped for roughly eight hours and nothing reported a problem. Cause: `_register_hook` runs `run_startup_diagnostic`, which on success does `get_default_publisher().write({'bootstrap_servers': brokers})`. Every pod writes that same single row at startup, so a multi-pod rollout has them all contending for it. The cron pod lost that race during the 6.90.20 rollout and `psycopg2.errors.SerializationFailure` aborted the registry load: ERROR odoo.modules.registry: Failed to load registry CRITICAL odoo.service.server: Failed to initialize database `bifrost`. The existing try/except around the diagnostic could not catch it. `write()` only queues the change; the UPDATE is issued later by `load_modules`' own `env.flush_all()`, outside that handler. The pod then stayed alive with no registry, executing nothing, while the liveness probe -- `pgrep -f /usr/bin/odoo` -- kept passing, because a process did exist. Kubernetes reported it healthy with restarts=0. 47 of 69 active crons went overdue; the most recent run was 8h20m old when it was found by accident. Two fixes. 1. Make the write unable to break startup. It is purely diagnostic -- `_get_kafka_producer` reads KAFKA_BROKERS from the environment and never consults the column. Skip it when the value is already correct, which is the normal restart and queues no UPDATE at all; and when it does change, write inside a savepoint and flush immediately so a lost race is caught and logged here instead of surfacing during flush_all(). 2. Make liveness test the job rather than the process. The probe now asks whether any active cron has run recently; a pod with no registry never advances ir_cron.lastcall, so the gap grows and the kubelet restarts it. It fails OPEN on anything unknown -- database unreachable, no psql, unparseable -- because a probe that restarts pods during a database incident turns one outage into two, and this instance hit storage-full on 2026-08-04. Verified against the live production cron pod: healthy -> 0; 480 minutes idle against a 120 limit -> 1 with a diagnostic line; 480 against a 600 limit -> 0; database unreachable -> 0. Chart invariants extended so a bare pgrep or a lost fail-open cannot come back, and they pass. lastCommitSHA: 985802abbc51daffd2ce0461a81f93ba0b7fa0d5 message: 'Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0' pipeline: greencapitaltrade/bifrost/main releaseNotesURL: https://github.com/greencapitaltrade/bifrost/releases/tag/v6.92.0 startedTimestamp: "2026-08-08T13:10:29Z" status: Succeeded steps: - kind: Stage stage: completedTimestamp: "2026-08-08T13:17:40Z" name: from build pack startedTimestamp: "2026-08-08T13:10:34Z" status: Succeeded steps: - completedTimestamp: "2026-08-08T13:10:55Z" name: Git Clone startedTimestamp: "2026-08-08T13:10:34Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:01Z" name: Next Version startedTimestamp: "2026-08-08T13:10:55Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:28Z" name: Jx Variables startedTimestamp: "2026-08-08T13:11:23Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:31Z" name: Setup Pip Cache startedTimestamp: "2026-08-08T13:11:28Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:32Z" name: Process Config Templates startedTimestamp: "2026-08-08T13:11:31Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:32Z" name: Update Asset Management Version startedTimestamp: "2026-08-08T13:11:32Z" status: Succeeded - completedTimestamp: "2026-08-08T13:11:37Z" name: Check Registry startedTimestamp: "2026-08-08T13:11:33Z" status: Succeeded - completedTimestamp: "2026-08-08T13:15:29Z" name: Build Container Build startedTimestamp: "2026-08-08T13:11:38Z" status: Succeeded - completedTimestamp: "2026-08-08T13:15:40Z" name: Promote Changelog startedTimestamp: "2026-08-08T13:15:30Z" status: Succeeded - completedTimestamp: "2026-08-08T13:15:44Z" name: Promote Helm Release startedTimestamp: "2026-08-08T13:15:40Z" status: Succeeded - completedTimestamp: "2026-08-08T13:16:52Z" name: Promote Jx Promote startedTimestamp: "2026-08-08T13:15:44Z" status: Succeeded - completedTimestamp: "2026-08-08T13:17:40Z" name: Publish Org Kb startedTimestamp: "2026-08-08T13:16:53Z" status: Succeeded - kind: Promote promote: completedTimestamp: "2026-08-08T13:16:17Z" environment: staging pullRequest: completedTimestamp: "2026-08-08T13:16:17Z" pullRequestURL: https://github.com/greencapitaltrade/mcu/pull/5297 startedTimestamp: "2026-08-08T13:16:17Z" status: Succeeded startedTimestamp: "2026-08-08T13:16:17Z" status: Succeeded - kind: Promote promote: completedTimestamp: "2026-08-08T13:16:49Z" environment: production pullRequest: completedTimestamp: "2026-08-08T13:16:49Z" pullRequestURL: https://github.com/greencapitaltrade/mcu/pull/5298 startedTimestamp: "2026-08-08T13:16:49Z" status: Succeeded startedTimestamp: "2026-08-08T13:16:49Z" status: Succeeded version: 6.92.0 status: {}