Bring your own generation model

By default, AI Search uses a Workers AI model to generate responses. To use a model outside of Workers AI, use AI Search for search and pass the retrieved content to a different model for generation. This guide uses an OpenAI model.

Prerequisites

Sign up for a Cloudflare account ↗.
Install Node.js ↗.

Node.js version manager

Use a Node version manager like Volta ↗ or nvm ↗ to avoid permission issues and change Node.js versions. Wrangler, discussed later in this guide, requires a Node version of 16.17.0 or later.

You also need:

An AI Search instance that already contains indexed content. To create one and add content, refer to Get started.
An OpenAI API key ↗.

1. Create a Worker project

Create a new Worker project using the create-cloudflare CLI (C3). C3 ↗ is a command-line tool designed to help you set up and deploy new applications to Cloudflare.

Create a new project named byo-model by running:


								
									
									npm
									 create cloudflare@latest -- byo-model


								
									
									yarn
									 create cloudflare byo-model


								
									
									pnpm
									 create cloudflare@latest byo-model


								
									
									bun
									 create cloudflare@latest byo-model

For setup, select the following options:

For What would you like to start with?, choose Hello World example.
For Which template would you like to use?, choose Worker only.
For Which language do you want to use?, choose TypeScript.
For Do you want to use git for version control?, choose Yes.
For Do you want to deploy your application?, choose No (we will be making some changes before deploying).

Go to your application directory:

cd byo-model

2. Install the AI SDK and OpenAI provider

Install the AI SDK ↗ and its OpenAI provider:


								
									
									npm
									 i ai @ai-sdk/openai


								
									
									yarn
									 add ai @ai-sdk/openai


								
									
									pnpm
									 add ai @ai-sdk/openai


								
									
									bun
									 add ai @ai-sdk/openai

3. Bind your Worker and set your API key

Add the AI Search binding to your Wrangler configuration file:

{
  "$schema": "./node_modules/wrangler/config-schema.json",
  "ai_search_namespaces": [
    {
      "binding": "AI_SEARCH",
      "namespace": "default",
      "remote": true
    }
  ]
}

[[ai_search_namespaces]]
binding = "AI_SEARCH"
namespace = "default"
remote = true

Store your OpenAI API key as a secret:

npx wrangler secret put OPENAI_API_KEY

For local development, add the key to a .dev.vars file in your project root instead:

.dev.varstxt

OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"

4. Add the code

Update src/index.ts. This Worker searches your instance, formats the retrieved chunks, and passes them to OpenAI to generate an answer. Replace my-instance with the name of your instance.

src/index.jsjs

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

export default {
	async fetch(request, env) {
		const url = new URL(request.url);
		const userQuery = url.searchParams.get("query") ?? "What is Cloudflare?";

		// Search for documents in AI Search.
		const searchResult = await env.AI_SEARCH.get("my-instance").search({
			messages: [{ role: "user", content: userQuery }],
		});

		if (searchResult.chunks.length === 0) {
			return Response.json({ text: `No data found for query "${userQuery}"` });
		}

		// Join the retrieved chunks into a single string.
		const chunks = searchResult.chunks
			.map((chunk) => `<file name="${chunk.item.key}">${chunk.text}</file>`)
			.join("\n\n");

		// Send the query and retrieved content to OpenAI for the answer.
		const openai = createOpenAI({ apiKey: env.OPENAI_API_KEY });
		const generateResult = await generateText({
			model: openai("gpt-4o-mini"),
			messages: [
				{
					role: "system",
					content:
						"You are a helpful assistant. Answer the user question using the provided files.",
				},
				{ role: "user", content: chunks },
				{ role: "user", content: userQuery },
			],
		});

		return Response.json({ text: generateResult.text });
	},
};

src/index.tsts

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

export interface Env {
	AI_SEARCH: AiSearchNamespace;
	OPENAI_API_KEY: string;
}

export default {
	async fetch(request, env): Promise<Response> {
		const url = new URL(request.url);
		const userQuery = url.searchParams.get("query") ?? "What is Cloudflare?";

		// Search for documents in AI Search.
		const searchResult = await env.AI_SEARCH.get("my-instance").search({
			messages: [{ role: "user", content: userQuery }],
		});

		if (searchResult.chunks.length === 0) {
			return Response.json({ text: `No data found for query "${userQuery}"` });
		}

		// Join the retrieved chunks into a single string.
		const chunks = searchResult.chunks
			.map((chunk) => `<file name="${chunk.item.key}">${chunk.text}</file>`)
			.join("\n\n");

		// Send the query and retrieved content to OpenAI for the answer.
		const openai = createOpenAI({ apiKey: env.OPENAI_API_KEY });
		const generateResult = await generateText({
			model: openai("gpt-4o-mini"),
			messages: [
				{
					role: "system",
					content:
						"You are a helpful assistant. Answer the user question using the provided files.",
				},
				{ role: "user", content: chunks },
				{ role: "user", content: userQuery },
			],
		});

		return Response.json({ text: generateResult.text });
	},
} satisfies ExportedHandler<Env>;

5. Run and deploy

Start a local development server, then query it at /?query=your+search+terms:

npx wrangler dev

npx wrangler login
npx wrangler deploy