Find integration setup failures

Show where customers fail while connecting integrations so engineering and CS can fix the highest-impact setup blockers.

Published

difficulty intermediate · time to value 5 minutes
#analytics #errors #integrations #onboarding #customer-success #engineering

Start from this 

Find integration setup failures by integration, setup step, account, and error code. Prioritize blockers affecting the most accounts.

Why this matters

Integrations are often the real activation moment for B2B and devtool products. A failed setup is not just a bug; it is a stalled customer.

What you get

  • Setup failure rate by integration
  • The step and error code causing the most blocked accounts
  • A short list of accounts CS should help
  • Engineering actions ranked by affected accounts

Walk through it

You

Which integrations are failing during setup?

Agent

I’ll compare setup starts, failures, and completions by integration and setup step.

Endpoint: POST /v1/projects/:project_id/analytics/query
Exact shape: /v1/openapi.json?path=%2Fv1%2Fprojects%2F%7Bproject_id%7D%2Fanalytics%2Fquery&method=post
Custom HogQL goes in the OpenAPI-defined `query` field.
Concept fields:
  query: "SELECT properties.integration_type AS integration, properties.setup_step AS step, properties.error_code AS error_code, count() AS failures, count(DISTINCT properties.account_id) AS accounts FROM events WHERE event = 'integration_setup_failed' AND timestamp > now() - INTERVAL 30 DAY GROUP BY integration, step, error_code ORDER BY accounts DESC, failures DESC LIMIT 20"
{
  "rows": [
    { "integration": "salesforce", "step": "oauth_callback", "error_code": "invalid_scope", "failures": 42, "accounts": 17 }
  ]
}
Agent

Salesforce OAuth scope failures are blocking 17 accounts. I’ll check whether any open cases match this pattern.

Endpoint: GET /v1/projects/:project_id/cases
Exact shape: /v1/openapi.json?path=%2Fv1%2Fprojects%2F%7Bproject_id%7D%2Fcases&method=get
Use query parameters, not a JSON body.
Query/filter concept:
  status: "open"

The output

The agent returns the integrations, setup steps, error codes, and affected-account counts that deserve action first. It should separate product-copy fixes from backend defects and CS follow-up.

Setting it up

Emit setup lifecycle events for each integration. Include the account, integration name, setup step, and normalized error code so failures can be grouped cleanly.

Variations

  • “Which integration setup failures affect enterprise accounts?”
  • “Show failed setup by integration version.”
  • “Draft a CS follow-up list for blocked accounts.”

Related playbooks

growth 5 minutes

Which doc page leads to the most API adoption?

Funnel from a docs page view to first api_request by the same distinct_id within 7 days. Per-doc-page breakdown — find the highest-leverage page in your docs.

debugging 30 seconds

Diagnose a customer complaint in 30 seconds

Paste a customer message. The agent pulls their session, finds the error they hit, walks their funnel, and tells you what to reply.

debugging 5 minutes

Find slow API routes by customer and plan

Rank API endpoints by p95 latency, error rate, customer, and plan so engineering can fix performance where it affects revenue.

Adapt this playbook in your own agent.

Ask your agent to map the starter prompt to your saved signal map, live events, and the current HTTP API before answering.

Set up Agentry in this repo by following https://agentry.sh/install.md. I authorize you to: run its device-code login and save the key to ~/.agentry/credentials.json, read this repo, propose an instrumentation plan for my approval, and edit code after I approve it.

+ Full access
5.5 Extra High
  1. 1. Open your repo in Codex, Claude Code, Cursor etc.
  2. 2. Paste the install prompt.
  3. 3. Your agent reads the install doc and shows you an implementation plan for approval.
← all playbooks