Talk to sales
Workbook

Analytics Engineer

You own: the metric layer — dbt-style models that turn raw data into trusted, documented metrics. With GeneFlow you also own the ML metric layer (which metrics describe model health) and how cost rolls up.

What's new for you

WasNow in GeneFlow
Metric definitions in dbtSame — plus GeneFlow exposes cost_usd, tokens_used as roll-up dimensions
ML metrics in a separate worldAll gf_metrics queryable via your warehouse export
No standard for "model freshness"Endpoint last_deployed_at, drift alert open status

Day-1 setup

Your warehouse should be replicating GeneFlow's gf_ tables (we publish them via CDC). Once they're in your warehouse you can dbt them like any source.

Workflow 1 — Build a "model production status" dbt model

-- models/marts/ml/dim_endpoints.sql
SELECT
  e.id                                    AS endpoint_id,
  e.tenant_id,
  e.name                                  AS endpoint_name,
  e.model_name,
  e.model_version,
  e.status                                AS endpoint_status,
  e.instance_type,
  e.replicas,
  e.hourly_cost_usd,
  e.total_cost_usd,
  e.drift_check_enabled,
  e.last_deployed_at,
  -- last 1h health
  (SELECT COUNT(*) FROM gf_inference_logs l
     WHERE l.endpoint_id = e.id AND l.ts >= now() - INTERVAL '1 hour') AS reqs_1h,
  (SELECT COUNT(*) FROM gf_inference_logs l
     WHERE l.endpoint_id = e.id AND l.ts >= now() - INTERVAL '1 hour' AND l.status_code >= 400) AS errs_1h,
  -- open drift alerts
  (SELECT COUNT(*) FROM gf_drift_alerts a
     WHERE a.endpoint_id = e.id AND a.acknowledged_at IS NULL) AS open_drift_alerts
FROM {{ source('geneflow', 'gf_endpoints') }} e
WHERE status <> 'DELETED'

Workflow 2 — Cost dimension

-- models/marts/cost/fct_geneflow_cost_daily.sql
WITH run_cost AS (
  SELECT tenant_id, date_trunc('day', start_time) AS day,
         SUM(cost_usd) AS run_cost_usd
  FROM {{ source('geneflow', 'gf_runs') }}
  GROUP BY 1, 2
),
inf_cost AS (
  SELECT tenant_id, date_trunc('day', ts) AS day,
         SUM(cost_usd) AS inference_cost_usd,
         COUNT(*) AS inferences
  FROM {{ source('geneflow', 'gf_inference_logs') }}
  GROUP BY 1, 2
),
endpoint_cost AS (
  SELECT tenant_id, date_trunc('day', last_update_at) AS day,
         SUM(hourly_cost_usd * 24) AS endpoint_idle_cost_usd
  FROM {{ source('geneflow', 'gf_endpoints') }}
  GROUP BY 1, 2
)
SELECT
  COALESCE(r.tenant_id, i.tenant_id, e.tenant_id) AS tenant_id,
  COALESCE(r.day, i.day, e.day) AS day,
  COALESCE(r.run_cost_usd, 0)        AS run_cost_usd,
  COALESCE(i.inference_cost_usd, 0)  AS inference_cost_usd,
  COALESCE(e.endpoint_idle_cost_usd, 0) AS endpoint_idle_cost_usd,
  COALESCE(r.run_cost_usd, 0) +
  COALESCE(i.inference_cost_usd, 0) +
  COALESCE(e.endpoint_idle_cost_usd, 0) AS total_geneflow_cost_usd,
  COALESCE(i.inferences, 0)          AS inferences
FROM run_cost r
FULL JOIN inf_cost i USING (tenant_id, day)
FULL JOIN endpoint_cost e USING (tenant_id, day)

Workflow 3 — Promote a metric to the platform's metric catalog

Use the existing /admin/metric-definitions UI to publish:

name: model_health.production_endpoints_with_open_drift
description: Production endpoints that currently have unacked drift alerts
type: gauge
sql: |
  SELECT COUNT(*) FROM dim_endpoints WHERE open_drift_alerts > 0
owner: ae@yourco.com
tags: [ml, governance]

Workflow 4 — Cost attribution per team

WITH owners AS (
  SELECT name, tags->>'team' AS team
  FROM gf_registered_models WHERE tenant_id='ACME'
)
SELECT o.team,
       SUM(r.cost_usd) AS train_cost,
       SUM(l.cost_usd) AS infer_cost
FROM owners o
JOIN gf_runs r            ON r.tags->>'model' = o.name
LEFT JOIN gf_endpoints e  ON e.model_name = o.name
LEFT JOIN gf_inference_logs l ON l.endpoint_id = e.id
WHERE r.start_time >= now() - INTERVAL '30 days'
GROUP BY o.team
ORDER BY train_cost + infer_cost DESC;

Common gotchas

  • gf_metrics is time-series — one row per (run, key, step). Don't fan out joins; pre-aggregate with MAX or LAST_VALUE.
  • Metric naming — agree a convention so DS/MLE don't both log auc with different definitions.

Where to go next