Custom engines

The plugin uses an engine abstraction for all AI operations. The default OpenAiEngine ships with the plugin, but you can build custom engines for any provider -- Anthropic, Azure OpenAI, Llama, on-prem solutions, or anything else that speaks HTTP.

BaseEngine contract

Extend the BaseEngine abstract class to create a custom engine. Four methods are required:

Method	Returns	Purpose
`chat(messages, options)`	`Promise<ChatResult>`	Core chat completion.
`generateImage(options)`	`Promise<GenerateImageResult>`	Image generation.
`getFunctionHandlers()`	`Record<string, Command>`	Command handler map for tool execution.
`getInfo()`	`Record<string, unknown>`	Engine metadata for diagnostics.

Optional overrides

Method	Default	Purpose
`init()`	No-op	Async initialization (auth, warmup, resource loading).
`supportedFeatures()`	`{ voice: false, voiceTranscription: false, vision: false }`	Feature capability flags.
`startVoiceSession(options)`	Throws	Start a realtime voice session.
`stopVoiceSession(handle)`	Throws	Stop a voice session.
`summarizeToolResult(params)`	Pass-through	Summarize tool execution results for follow-up prompts.
`getMCPTools()`	`[]`	Return configured MCP server tools.

Minimal example

custom.tsCustom engine

ai.mjsRegister engine

src/engines/custom.ts

import { BaseEngine } from '@robojs/ai'
import type {
	ChatMessage,
	ChatOptions,
	ChatResult,
	GenerateImageOptions,
	GenerateImageResult
} from '@robojs/ai'

export class CustomEngine extends BaseEngine {
	async init() {
		// Authenticate, warm up connections
	}

	async chat(messages: ChatMessage[], options: ChatOptions): Promise<ChatResult> {
		const response = await myProvider.complete(messages)
		return {
			finish_reason: 'stop',
			message: { role: 'assistant', content: response.text }
		}
	}

	async generateImage(options: GenerateImageOptions): Promise<GenerateImageResult> {
		throw new Error('Image generation not supported')
	}

	getFunctionHandlers() {
		return {}
	}

	getInfo() {
		return { name: 'custom', version: '1.0.0' }
	}

	supportedFeatures() {
		return { voice: false, voiceTranscription: false, vision: true }
	}
}

src/engines/custom.js

import { BaseEngine } from '@robojs/ai'

export class CustomEngine extends BaseEngine {
	async init() {
		// Authenticate, warm up connections
	}

	async chat(messages, options) {
		const response = await myProvider.complete(messages)
		return {
			finish_reason: 'stop',
			message: { role: 'assistant', content: response.text }
		}
	}

	async generateImage(options) {
		throw new Error('Image generation not supported')
	}

	getFunctionHandlers() {
		return {}
	}

	getInfo() {
		return { name: 'custom', version: '1.0.0' }
	}

	supportedFeatures() {
		return { voice: false, voiceTranscription: false, vision: true }
	}
}

Registration

Wire your engine in the plugin config:

config/plugins/robojs/ai.ts

import { CustomEngine } from '../src/engines/custom.js'

export default {
	engine: new CustomEngine()
}

config/plugins/robojs/ai.mjs

import { CustomEngine } from '../src/engines/custom.js'

export default {
	engine: new CustomEngine()
}

When a custom engine is provided, OPENAI_API_KEY isn't required. The engine handles its own authentication.

ChatResult

The chat() method must return a ChatResult object:

Prop

Type

EngineSupportedFeatures

Feature flags returned by supportedFeatures():

Prop

Type

Voice-capable engines

To support voice, implement these methods and set supportedFeatures().voice to true:

startVoiceSession(options: VoiceSessionStartOptions) -- Returns a VoiceSessionHandle with pump(), commitInput(), and stop() methods for controlling the audio stream.
stopVoiceSession(handle: VoiceSessionHandle) -- Cleans up the voice session and releases resources.

The VoiceSessionHandle provides:

Prop

Type

Next steps

Hooks

Intercept and customize the AI pipeline.

Configuration

All plugin and engine options.

Custom engines

Hooks

Configuration

On this page