Test your app
Tests that keep the guarantees of the policy and the row rules - no raw functions, no leaks between tenants, a budget for each call - and tests of scheduled work and agents.
The policy and the row rules protect every function that goes through
defineFunctions. Three tests keep it that way while the application grows:
no raw functions, no leaks between tenants, and a budget for each call. The
last two sections test scheduled work and agents. All tests run in
convex-test with the real Better Auth component, so nothing is mocked except
the deployment and outside services.
The first sections show the tests of
starters/mcp-oauth-agent:
the short files in full, and the shape of the long ones with a link to the
file. The tests of the last two sections use the app of
Start here and
Internal operations.
Set up convex-test
pnpm add -D convex-test vitestimport { defineConfig } from 'vitest/config'
export default defineConfig({
test: { environment: 'node', include: ['convex/**/*.test.ts'] },
})/// <reference types="vite/client" />
import betterAuth from '@lupinum/better-convex-nuxt/better-auth/test'
import { convexTest } from 'convex-test'
import { vi } from 'vitest'
import schema from './schema'
export const modules = import.meta.glob('./**/*.ts', { eager: false })
/** convex-test with the real Better Auth component. */
export function initConvexTest() {
// The issuer and the resource of an MCP grant (`grantMcp`) come from these.
vi.stubEnv('SITE_URL', 'https://app.example.test')
vi.stubEnv('CONVEX_SITE_URL', 'https://deployment.example.test')
const t = convexTest(schema, modules)
betterAuth.register(t)
return t
}Convex does not deploy files with two dots in the name, such as
test.setup.ts and leaks.test.ts.
The tests import ./_generated/api. After you add a module or an export, run
pnpm exec better-convex-nuxt convex codegen. It needs a deployment; a local one
works without an account or network. See
Code generation without a cloud deployment.
signInAs(t, 'ann') from @lupinum/better-convex-nuxt/better-auth/test
returns a client that calls functions as that person. It creates two rows in
the Better Auth component, or reuses them when they exist:
| Row | ID | Other fields |
|---|---|---|
| user | ann | name ann, email ann@example.com |
| session | ann-session | expires in one hour (expiresInMs option) |
Insert your app's own users row yourself, with authId: 'ann'. When the
session was deleted, for example to test a sign-out, the next signInAs
creates a session with a new ID.
The session expires one hour after signInAs, also in fake time. With
vi.useFakeTimers(), t.finishAllScheduledFunctions(vi.runAllTimers) runs
every timer, so it also moves the clock past the session's end, and the next
call is signed out. Move the clock in small steps instead, or give the session
a longer life. Pass expiresInMs on the first signInAs for that person in the
test: a later call reuses the live session and keeps its expiry.
// Run the scheduled work of the next 10 seconds, one second at a time. Work
// that a step schedules starts in a later step, so take a few steps more than
// the time of the last job.
for (let step = 0; step < 12; step += 1) {
vi.advanceTimersByTime(1_000)
await t.finishInProgressScheduledFunctions()
}
// Or: a session that outlives a job scheduled 24 hours later.
const ann = await signInAs(t, 'ann', { expiresInMs: 7 * 24 * 60 * 60 * 1000 })The loop does not work for a scheduled action that awaits a timer itself, such
as setTimeout: the test waits for the action while the clock stands still. Use
t.finishAllScheduledFunctions(vi.runAllTimers) with a longer session there.
Assert codes, not messages
A refused call rejects with a ConvexError whose data.code says why:
NOT_FOUND, FORBIDDEN, APPROVAL_NOT_FOUND. Compare the code with one
literal value. The codes are part of the public API. The message texts are
not, and a release can change them.
No raw functions
A function built with Convex's own query, mutation, internalMutation or
httpAction gets no actor, no policy and no row rules. An operation that
calls a raw internal function fails, but a raw public function is open to
every caller. This test finds each one.
import { unguardedFunctions } from '@lupinum/better-convex-functions/test'
import { expect, test } from 'vitest'
import { modules } from './test.setup'
// Catches a function built with Convex's own builders instead of the ones from ./functions:
// it would skip the caller check, the policy and the row rules. Mark a deliberate one with
// `trusted(reason, fn)`.
test('every function goes through the policy and the row rules', async () => {
const trustedRoutes = { '/api/auth/': 'Better Auth endpoints; the auth library checks each one.' }
expect(await unguardedFunctions(modules, { trustedRoutes })).toEqual([])
})unguardedFunctions takes the module map that convexTest receives, so it
checks the modules that Convex deploys. Like Convex, it skips files with two
dots in the name, _generated/ at the root, and a local component (a folder
with its own convex.config.ts). It throws when the map is empty or has no
defineFunctions operation, because then the test would check nothing.
When a function must skip the library, for example an operator function or
a trigger of the auth component, wrap it and say why:
trusted('Operator function: run it from the dashboard.', internalMutation({ … })).
The reason shows up when someone reviews the list of trusted functions.
convex/connections.ts and convex/auth.ts in the starter show both kinds.
trustedRoutes does the same for HTTP routes under a path prefix that
another library checks, such as the Better Auth endpoints. A prefix covers
whole path segments: /api/auth covers /api/auth/session, not
/api/authz/users.
Run this test in CI. It is the guarantee: the runtime checks are the second line. See Threat model.
Launch check
launchProblems runs the checks that matter most before an app goes live. It
returns one plain sentence for each problem, and an empty list when all is well.
import { launchProblems } from '@lupinum/better-convex-functions/test'
import { expect, test } from 'vitest'
import crons from './crons'
import { fns } from './functions'
import schema from './schema'
import { modules } from './test.setup'
// Catches the mistakes that hurt after launch: a raw Convex function, a table that keeps a
// person's data after they delete their account, housekeeping nobody schedules, a public write
// without a limit, and a tool that no agent sees. Each problem names the fix.
test('the app has no launch problems', async () => {
const trustedRoutes = { '/api/auth/': 'Better Auth endpoints; the auth library checks each one.' }
expect(await launchProblems({ modules, schema, crons, fns, trustedRoutes })).toEqual([])
})It checks five things, and nothing else:
- Raw functions. The same check as
unguardedFunctions.trustedRoutesis passed on. - Account deletion. Every field that holds
v.id('users')is covered by an entry in theerasuremap ofdefineFunctions(or the table haskeep), and a module exportsfns.erasure.eraseStep. The check also finds an ID inside an object, an array or a record (as key or value). Erasure cannot reach such an ID, so onlykeepcovers it, or you store the ID in its own indexed field. The check looks forv.id('users'), so the table of your users must be calledusers. That table and the library tables need no entry. Withouterasure, the sentence says that account deletion is not set up. - Housekeeping. If the agents package built a housekeeping function in your modules, a cron in
cronscalls it. - Limits on public writes. Every public action that a public mutation uses has a
limitsentry in the policy. - Tools in a catalog. Every operation with a
toolis in a module that you pass todefineTools, and the module that callsdefineToolsexports its result astools. Otherwise no agent sees the tool.
Each sentence names the file, table or action and the fix. To accept a finding
on purpose, use the way that exists already: trusted(reason, fn),
erasure: { table: { keep: reason } }, or a limit.
It does not check that your rules, roles or limits are the right ones, that
beforeDelete refuses what it should, or anything outside Convex, such as
files in storage or other services.
No leaks between tenants
Put one row of another tenant in the database: the canary. Then let people who must not reach it aim every call at it. Each row of the table names an actor, a call and the one code the call must fail with:
| Actor | Calls | Code |
|---|---|---|
| A stranger: no membership in the tenant | every call | NOT_FOUND |
A former member: membership status removed | every call | NOT_FOUND |
| A lower role: a viewer in the tenant | every write | FORBIDDEN |
| Each of them, through their own agent | check_approval with the ID of another agent's request | APPROVAL_NOT_FOUND |
A stranger and a former member get NOT_FOUND, not FORBIDDEN: the answer
must not tell them that the row exists.
The starter's
convex/leaks.test.ts
has one call for each operation, through the web client, and one for each
tool, through callTool (see Tools and approvals).
Each call takes the IDs it aims at:
const operations: Record<Exclude<keyof typeof projects, 'organizations'>, Call> = {
search: (_t, eve, { organizationId }) =>
eve.web.query(api.projects.search, { organizationId, paginationOpts }),
rename: (_t, eve, { projectId }) =>
eve.web.mutation(api.projects.rename, { projectId, name: 'Mine' }),
// … one entry for each operation
}
const toolCalls: Record<string, Call> = {
archive_project: (t, eve, { projectId }) =>
callTool(t, tools, eve.principal, 'archive_project', { projectId }),
// … one entry for each tool that takes an ID
}
const rows: [Actor, string, string][] = [
['stranger', 'rename', 'NOT_FOUND'],
['former member', 'archive_project', 'NOT_FOUND'],
['viewer', 'rename', 'FORBIDDEN'],
// … one row for each actor and call
]Each row runs three checks:
it.each(rows)('a %s: %s fails with %s', async (actor, name, expected) => {
const { t, bob, eve } = await setup(actor)
const call = calls[name]!
const before = await state()
await expect(call(t, eve, bob).then(() => 'no error', code)).resolves.toBe(expected)
// Nothing changed: no project renamed, archived or added, and no new request for a person.
await expect(state()).resolves.toEqual(before)
// The positive control: the same call through the same path succeeds with Eve's own IDs.
await expect(call(t, eve, eve)).resolves.toBeDefined()
})- One literal code. A check such as "the result does not name the
canary" passes for any refusal, also for
INVALID_INPUTfrom a wrong argument name. Then the call never reached the tenant check. - The positive control. The same call with the actor's own IDs must succeed. It proves that the call is correct and that only the foreign ID makes it fail.
- Nothing changed. The canary is unchanged, no row was added to the other tenant, and no approval request waits there.
Test the tools separately. A tool runs the same operation, but an agent's call takes another path: a write that needs approval becomes a request for a person, and a request for another tenant's row must fail before it reaches that tenant's approval queue.
Record<Exclude<keyof typeof projects, 'organizations'>, …> makes the test
fail to compile when a new operation has no call. organizations takes no ID;
the same file checks that it lists only the organizations of active
memberships. For the tools, one test compares the call names with
tools.catalog.
keyof typeof names every export of the module, also internal operations,
which a person cannot call. Keep internal operations in a module of their own,
as convex/summaryWork.ts in
Internal operations. When a module
must hold both, leave the internal ones out by name. For a sites module with
the internal operations runCheck and saveResult:
Record<Exclude<keyof typeof sites, 'runCheck' | 'saveResult'>, …>.
When an operation takes two IDs, add a row that mixes them: one of the actor's own tenant and one of the canary's.
A budget for each call
countDocuments from @lupinum/better-convex-functions/test counts the
documents one call reads and writes, with everything the library does around
the handler: the session, the user, the role and the tenant rows. The
starter's
convex/cost.test.ts
sets up one organization with three projects and counts two calls:
// Catches a change that makes a call read or write more than it did: an extra lookup per row,
// a missing index, a write in a read path. Each number is the budget; raise it on purpose.
test('a page of three projects', async () => {
const { ann, organizationId } = await setup()
const cost = await countDocuments(() =>
ann.query(api.projects.search, {
organizationId,
paginationOpts: { numItems: 10, cursor: null },
}),
)
// The session check (2), user, organization, membership, then the three rows of the page.
expect(cost).toEqual({ reads: 8, writes: 0 })
})Each number is a budget. A change that reads more, for example a lookup per row instead of one per call, fails the test and must raise the number on purpose. Cost lists what each kind of call costs.
countDocuments follows convex-test: a query counts the rows it returns, not
the rows that a .filter() skipped. A deployment counts those too. Do not run
calls in parallel while you count.
The counts include the library's own reads, so they can change when you upgrade Better Convex. The changelog says so under "Tests". Check the new numbers and update the literals.
Scheduled functions
convex-test runs scheduled functions with setTimeout. Fake only
setTimeout and clearTimeout, and keep the real clock: with a fake clock,
running the timers moves the time past the expiry of the session and of an
agent's grant, and the scheduled call fails with NOT_SIGNED_IN or
AGENT_DISABLED.
import { signInAs } from '@lupinum/better-convex-nuxt/better-auth/test'
import { afterEach, beforeEach, expect, test, vi } from 'vitest'
import { api } from './_generated/api'
import { initConvexTest } from './test.setup'
beforeEach(() => {
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] })
// The summary service is outside the app: answer for it.
vi.stubGlobal(
'fetch',
vi.fn(async () => Response.json({ summary: 'Plans for the next year.' })),
)
})
afterEach(() => {
vi.useRealTimers()
vi.unstubAllGlobals()
})
/** Runs scheduled work, and the work it schedules, until none is left. */
async function runScheduled(t: ReturnType<typeof initConvexTest>) {
for (let step = 0; step < 10; step++) {
if (vi.getTimerCount() === 0) return
vi.runOnlyPendingTimers()
await t.finishInProgressScheduledFunctions()
}
throw new Error('Scheduled work still runs after 10 steps.')
}
// Catches a broken chain from the request to the saved summary: the scheduled action, its read
// and its write, each as the person who asked.
test('a requested summary is saved on the project', async () => {
const t = initConvexTest()
const projectId = await t.run(async (ctx) => {
const userId = await ctx.db.insert('users', { authId: 'ann', name: 'Ann' })
const organizationId = await ctx.db.insert('organizations', { name: 'Acme' })
await ctx.db.insert('memberships', { organizationId, userId, role: 'owner' })
return await ctx.db.insert('projects', { organizationId, name: 'Roadmap', status: 'active' })
})
const ann = await signInAs(t, 'ann')
await ann.mutation(api.summaries.request, { projectId })
await runScheduled(t)
await expect(t.run((ctx) => ctx.db.get(projectId))).resolves.toMatchObject({
summary: 'Plans for the next year.',
})
})runScheduled starts what is scheduled, waits until it is done, and repeats
for the work that schedules more work. The step limit stops a chain that never
ends.
An agent's grant from grantMcp lasts 10 minutes, like an access token. For
scheduled work that runs longer, give the session and the grant a longer life:
await signInAs(t, 'ann', { expiresInMs: 4 * 60 * 60 * 1000 }) // the session grantMcp reuses
const principal = {
...(await grantMcp(t, 'ann', ['projects:read'])),
expiresAt: Math.floor(Date.now() / 1000) + 4 * 60 * 60,
}Tools and approvals
grantMcp(t, 'ann', scopes) from @lupinum/better-convex-nuxt/better-auth/test
gives Ann a live MCP grant: a host client test-host, the resource
${CONVEX_SITE_URL}/mcp, and her consent to the scopes. It returns the
principal that the door passes to a tool after it checks an access token. The
grant uses the same user and session as signInAs, in either order, so
deleting the session ends both, as a sign-out does.
callTool(t, tools, principal, name, input) from
@lupinum/better-convex-agents/test calls a tool as the MCP door does: only a
tool that the principal's scopes unlock, with request_id taken from input.
The input goes through JSON, as a host's does: NaN arrives as null. It
resolves with the tool's output: { status: 'done', result }, or
{ status: 'needs_approval', approvalId, summary, url }. It rejects with the
tool's own error, as t.mutation does. A tool that the scopes do not unlock
throws an Error that lists the tools they unlock.
Three tests cover what only agents have.
The agent surface. One literal of every tool, its scopes and its approval rule. A scope that unlocks a write tool by accident, or an approval rule that was dropped, changes the literal. Review a change to it like a change to the policy. The test below is for the app of Start here.
An agent's request. It waits for a person, and the approval runs it.
import { callTool } from '@lupinum/better-convex-agents/test'
import { grantMcp, signInAs } from '@lupinum/better-convex-nuxt/better-auth/test'
import { expect, test } from 'vitest'
import { api } from './_generated/api'
import { tools } from './agents'
import { initConvexTest } from './test.setup'
// Catches a scope that unlocks a write tool by accident, and an approval rule that was dropped.
test('gives agents these tools, with these scopes and approvals', () => {
expect(
tools.catalog.map(({ name, kind, scopes, approval }) => ({ name, kind, scopes, approval })),
).toEqual([
{ name: 'list_projects', kind: 'query', scopes: ['projects:read'], approval: 'never' },
{ name: 'archive_project', kind: 'mutation', scopes: ['projects:write'], approval: 'always' },
{ name: 'check_approval', kind: 'query', scopes: [], approval: 'never' },
])
})
// Catches an agent that archives without a person, and an approval that does not run the archive.
test('an agent’s archive waits for a person, then runs as the agent', async () => {
const t = initConvexTest()
const projectId = await t.run(async (ctx) => {
const userId = await ctx.db.insert('users', { authId: 'ann', name: 'Ann' })
const organizationId = await ctx.db.insert('organizations', { name: 'Acme' })
await ctx.db.insert('memberships', { organizationId, userId, role: 'owner' })
return await ctx.db.insert('projects', { organizationId, name: 'Roadmap', status: 'active' })
})
const principal = await grantMcp(t, 'ann', ['projects:read', 'projects:write'])
const ann = await signInAs(t, 'ann')
const asked = await callTool(t, tools, principal, 'archive_project', { projectId })
if (asked.status !== 'needs_approval') throw new Error('The archive did not wait for a person.')
expect(asked.summary).toBe('Archive the project "Roadmap".')
await expect(t.run((ctx) => ctx.db.get(projectId))).resolves.toMatchObject({ status: 'active' })
await expect(ann.mutation(api.agents.approve, { approvalId: asked.approvalId })).resolves.toEqual(
{ status: 'approved' },
)
await expect(t.run((ctx) => ctx.db.get(projectId))).resolves.toMatchObject({
status: 'archived',
})
})Who may approve. A table of people who try to approve another person's
agent request, with the result for each. The starter's
convex/agents.test.ts
lets owners and admins decide (approvers: { 'projects.archive': ['owner', 'admin'] }):
it.each([
['an owner', 'approved', 'owner', 'active'],
['an admin', 'approved', 'admin', 'active'],
['a member', 'APPROVAL_NOT_FOUND', 'member', 'active'],
['a viewer', 'APPROVAL_NOT_FOUND', 'viewer', 'active'],
['a former admin', 'APPROVAL_NOT_FOUND', 'admin', 'removed'],
['a stranger', 'APPROVAL_NOT_FOUND', null, null],
] as const)('%s who approves gets %s', async (_name, expected, role, status) => {
// … Ann's agent asks to archive; Pat, with this role, tries to approve it.
expect(outcome).toBe(expected)
// A refused approval leaves the project as it was.
await expect(t.run((ctx) => ctx.db.get(projectId))).resolves.toMatchObject({
organizationId,
status: expected === 'approved' ? 'archived' : 'active',
})
})A principal with fewer scopes stands for an access token with fewer scopes:
grantMcp(t, 'ann', ['projects:read']). The starter's
convex/authorization.test.ts uses grantMcp and callTool to check that
each tool call stops after the app suspends the person or removes the
membership.