VelaVela
Cloudflare

Workers AI

Use Cloudflare Workers AI inference in your Vela application.

The AIModule provides a managed AIService for running AI models on Cloudflare's edge network.

Setup

Import the module and configure the binding name:

import { Module } from '@velajs/vela';
import { AIModule } from '@velajs/cloudflare';

@Module({
  imports: [AIModule.forRoot({ binding: 'AI' })],
})
class AppModule {}

Add the AI binding to wrangler.toml:

[ai]
binding = "AI"

Using AIService

Inject AIService to run inference:

import { Controller, Post, Body } from '@velajs/vela';
import { AIService } from '@velajs/cloudflare';

@Controller('ai')
class AIController {
  constructor(private readonly ai: AIService) {}

  @Post('chat')
  async chat(@Body() body: { prompt: string }) {
    const result = await this.ai.run('@cf/meta/llama-3-8b-instruct', {
      messages: [{ role: 'user', content: body.prompt }],
    });
    return result;
  }

  @Post('embed')
  async embed(@Body() body: { text: string[] }) {
    const result = await this.ai.run('@cf/baai/bge-base-en-v1.5', {
      text: body.text,
    });
    return result;
  }
}

Methods

MethodDescription
run(model, inputs, options?)Run a model. The model string identifies the model (e.g., @cf/meta/llama-3-8b-instruct). inputs vary by model type. Optional options can include gateway for AI Gateway integration.

Durable Objects

Use Cloudflare Durable Objects for stateful coordination in your Vela application.

Vectorize

Use Cloudflare Vectorize for vector search in your Vela application.

Edit on GitHub

Last updated on

On this page

SetupUsing AIServiceMethods