AI Business Maturity Model
Certifications
Find a CoachFind a SpeakerSign In
Deep Dive

/

Part 2

Part 2 of 6

The Tool System

The Employee's Hands

Tools are how the AI interacts with the real world. Every tool is a TypeScript function that takes structured parameters, queries a database or API, and returns a standardized result. The permission system ensures the AI can never modify data without human approval.

The ToolResult Contract

Every tool — regardless of what it does — returns this shape. The contract is the foundation of the tool system. It separates what the AI reads (data) from what the user sees (summary) from what gets injected into the next prompt (markdown).

lib/tools/types.ts

TypeScript
interface ToolResult {
  /** Whether the tool executed successfully */
  success: boolean;

  /**
   * Raw structured data for the AI to reason about.
   * This appears in the tool history injected into each pass's prompt.
   * Keep it structured — the AI reads JSON better than prose.
   */
  data?: unknown;

  /**
   * Short human-readable description shown in the UI activity panel.
   * Shown to the user while the tool is running and after it completes.
   * Example: "Found 47 records matching criteria"
   */
  summary: string;

  /**
   * Formatted Markdown output for prompt injection.
   * Optional — use when the data benefits from formatting.
   * Example: a Markdown table of search results.
   */
  markdown?: string;

  /** Error message if success is false */
  error?: string;
}
Example: A search tool result

Example ToolResult from data.searchRecords

TypeScript
// What the tool returns:
{
  success: true,
  summary: "Found 47 records matching criteria in Region A",
  data: {
    records: [
      { id: "rec-001", name: "Acme Corp", score: null, status: "active" },
      { id: "rec-002", name: "Beta LLC",  score: null, status: "active" },
      // ... 45 more
    ],
    total: 47,
    filters_applied: { region: "A", category: "enterprise", status: "active" }
  },
  markdown: `
## Search Results: Region A Enterprise Records
| ID | Name | Status |
|----|------|--------|
| rec-001 | Acme Corp | active |
| rec-002 | Beta LLC | active |
... (47 total)
  `
}

Permission Levels — The Safety Layer

Every tool has a permissionLevel that controls whether it auto-executes or requires user approval. This is the core safety mechanism. The AI can search and analyze freely, but it cannot create, update, or delete anything without a human clicking "Approve."

read
Yes, immediately

Safe, non-destructive queries. The AI can call these freely in any pass.

Examples: data.searchRecords, data.getStats, analysis.crossReference

draft
Yes, immediately

AI-generated content previews. Auto-execute because they produce drafts, not final changes.

Examples: content.generateSummary, report.buildPreview

write
Requires user click

Creates or modifies real data. The loop pauses and shows an approval card.

Examples: data.updateRecord, data.addToList, data.qualifyRecord

destructive
Always manual

Irreversible bulk operations. Never auto-execute under any circumstances.

Examples: data.bulkDelete, data.archiveAll

Tool Naming Convention

Tools use category.toolName dot notation. This makes the AI's tool calls readable and groups related tools visually in the system prompt's tool menu. Coulee Tech used this convention across 91 tools organized into 8 categories.

Tool naming examples

TypeScript
// Pattern: category.actionTarget

// Data access (read)
'data.searchRecords'      // Search with filters
'data.getRecordDetail'    // Get a single record
'data.getStats'           // Aggregate statistics

// Analysis (read/draft)
'analysis.crossReference' // Compare against another dataset
'analysis.scoreRecord'    // Score a single record
'analysis.batchScore'     // Score many records in one call

// Content (draft)
'content.generateSummary' // AI-generated summary
'content.buildReport'     // Build a formatted report

// Data mutations (write — require approval)
'data.updateRecord'       // Update a single record
'data.bulkUpdate'         // Update many records
'data.addToList'          // Add to a collection

// External (read)
'web.searchCompany'       // Web research
'web.validateDomain'      // Check if domain is active

How a Tool Is Executed

When the engine processes the AI's tool call requests, it goes through executeToolByName() in the tool registry. Here is the complete execution path for a single tool call:

lib/tools/registry.ts — executeToolByName

TypeScript
async function executeToolByName(
  toolName: string,
  params: Record<string, unknown>,
  context: ToolExecutionContext
): Promise<ToolResult> {
  // 1. Look up the tool in the registry map
  const tool = toolRegistry.get(toolName);
  if (!tool) {
    return { success: false, summary: `Tool not found: ${toolName}`, error: 'TOOL_NOT_FOUND' };
  }

  // 2. Check the Employee's allowedTools whitelist
  if (!context.allowedTools.includes(toolName)) {
    return { success: false, summary: `Tool not permitted: ${toolName}`, error: 'NOT_PERMITTED' };
  }

  // 3. Validate required parameters
  for (const requiredParam of tool.requiredParams) {
    if (params[requiredParam] === undefined) {
      return {
        success: false,
        summary: `Missing required param: ${requiredParam}`,
        error: 'MISSING_PARAM'
      };
    }
  }

  // 4. Execute the tool with injected context
  //    organizationId and userId come from the session — never from the request body
  const result = await tool.execute(params, {
    organizationId: context.organizationId,
    userId: context.userId,
  });

  return result;
}
Parallel execution via Promise.allSettled

All auto-execute tools in a single pass run simultaneously. A 5-tool pass takes as long as the slowest tool, not the sum of all tools. Error isolation means one failing tool doesn't crash the others.

Parallel tool execution in the engine

TypeScript
// All auto-execute tools run in PARALLEL
const results = await Promise.allSettled(
  autoExecuteTools.map(async (call) => {
    const startTime = Date.now();

    // Emit "tool starting" event to the UI
    yield { type: 'tool_start', data: { tool: call.tool, params: call.params } };

    const result = await executeToolByName(call.tool, call.params, context);
    const duration = Date.now() - startTime;

    // Record result in state (for prompt injection in next pass)
    recordToolResult(state, call.tool, call.params, result, duration);

    // Mark as executed (deduplication)
    markToolExecuted(state, call.tool, call.params);

    // Emit "tool done" event to the UI
    yield { type: 'tool_result', data: { tool: call.tool, result, duration } };
  })
);
// Promise.allSettled — one failure doesn't crash the rest

Deduplication — Never Run the Same Tool Twice

The engine maintains a Set<string> of executed tool signatures. A signature is toolName::JSON.stringify(params). If the AI requests a tool with identical params it already ran, the engine skips it silently. If all requested tools are duplicates, the loop exits.

Deduplication logic

TypeScript
// In state: executedToolSignatures = new Set<string>()

function isToolAlreadyExecuted(
  state: WorkerState,
  toolName: string,
  params: Record<string, unknown>
): boolean {
  const signature = `${toolName}::${JSON.stringify(params)}`;
  return state.executedToolSignatures.has(signature);
}

function markToolExecuted(
  state: WorkerState,
  toolName: string,
  params: Record<string, unknown>
): void {
  const signature = `${toolName}::${JSON.stringify(params)}`;
  state.executedToolSignatures.add(signature);
}

// In the execution loop:
for (const call of toolCalls) {
  if (isToolAlreadyExecuted(state, call.tool, call.params)) {
    skippedDuplicates.push(call.tool);
    continue; // Skip silently
  }
  // ... execute
}

// If ALL requested tools were duplicates, exit the loop
if (skippedDuplicates.length === toolCalls.length) {
  return { type: 'exit', reason: 'all_tools_duplicate' };
}

Building a Tool — The Template

Here is the generic template for a tool. Every tool in Coulee Tech's system followed this pattern — a class with metadata and an execute method.

lib/tools/data/searchRecords.ts

TypeScript
import type { ToolDefinition, ToolResult } from '../types';

export const searchRecordsTool: ToolDefinition = {
  name: 'data.searchRecords',
  description: 'Search and filter records by criteria',
  permissionLevel: 'read',
  requiredParams: [],  // All params optional for flexible search
  paramSchema: {
    region: { type: 'string', description: 'Filter by region code' },
    category: { type: 'string', description: 'Filter by category' },
    status: { type: 'string', description: 'Filter by status (active|inactive|pending)' },
    limit: { type: 'number', description: 'Max results (default: 100)' },
  },

  async execute(
    params: { region?: string; category?: string; status?: string; limit?: number },
    context: { organizationId: string; userId: string }
  ): Promise<ToolResult> {
    try {
      // organizationId is always injected — never trust params for scoping
      const records = await db.records.findMany({
        where: {
          organizationId: context.organizationId,
          ...(params.region && { region: params.region }),
          ...(params.category && { category: params.category }),
          ...(params.status && { status: params.status }),
        },
        take: params.limit ?? 100,
      });

      return {
        success: true,
        summary: `Found ${records.length} records${params.region ? ` in ${params.region}` : ''}`,
        data: {
          records: records.map(r => ({ id: r.id, name: r.name, status: r.status })),
          total: records.length,
          filters_applied: params,
        },
        markdown: buildMarkdownTable(records),
      };
    } catch (error) {
      return {
        success: false,
        summary: 'Search failed',
        error: error instanceof Error ? error.message : 'Unknown error',
      };
    }
  },
};

Part 1: Worker DefinitionPart 3: The Engine