Skip to content

Commit b6b9dca

Browse files
authored
chore(ai): remove redundant runtime type validation (#24095)
## Background - several runtime checks in AI SDK check for things that our TypeScript types already enforce - this only guards against users bypassing TypeScript by using plain javascript, `any`, or type assertions, which we shouldn't support - the previous `streamText` validation was expensive because it had to validate every message in history on every step ## Summary - Remove the `ModelMessage` schema validation in `standardizePrompt`. - Still check for empty messages and system messages, which types can't express - Remove type checks that duplicate the types on call settings - `temperature`, `topP`, `topK`, `presencePenalty`, `frequencyPenalty` being numbers - `smoothStream` chunking being one of the possible values - steering messages checking that role is user - tool caller config being an array with each caller being a string - `experimental_streamSpeech` throwing if input didn't match one of the possible inputs - `generateObject`/`streamObject` validating `output` types and requiring a `schema` when present - `decide` checking state types and question shapes - batch reference version being checked that it's `2` - In `decide`, keep only the checks that types cannot express (non-finite numbers, array holes, etc). - Updated `InvalidPromptError` docs to mention current causes. Callers who pass malformed input without going through types will now get a `TypeError` from the runtime instead of an SDK validation error. Callers who respect the types see no change. The biggest benefit is not validating every message in `streamText` on every step anymore. With 2,201 messages - the amount from the original issue - and a mocked model that instantly returns: | Metric | main | this PR | delta | | --- | --- | --- | --- | | `streamText`, each later step | 37.5ms | 0.25ms | −37.3ms (~150× faster) | | `streamText`, first step | 74.5ms | 0.39ms | −74.1ms (~190× faster) | | `streamText`, 11-step run | 456.7ms | 3.9ms | −452.8ms (~115× faster) | | `generateText`, first step | 37.1ms | 0.33ms | −36.8ms (~110× faster) | | `generateText`, each later step | 0.22ms | 0.20ms | no change | | Single-message `generateText` call | 0.4ms | 0.4ms | no change | ## Related Issues - Fixed #24048 by just removing that validation altogether
1 parent 57cb73a commit b6b9dca

19 files changed

Lines changed: 133 additions & 509 deletions
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
'ai': patch
3+
---
4+
5+
chore(ai): remove runtime validation of inputs that TypeScript types already guarantee
6+
7+
`streamText`, `generateText`, `generateObject` and `streamObject` no longer validate messages against the `ModelMessage` schema. This removes a cost that grew linearly with message history and was paid on every `streamText` step. Type checks on call settings, steering messages, tool caller configuration, `smoothStream` chunking, `experimental_streamSpeech` text input, the shape of `decide` state and questions, and the batch reference version have also been removed. Value constraints that types cannot express, such as integer ranges, non-empty collections, unknown keys, and non-finite numbers or cycles in `decide` JSON input, are still validated.

‎content/docs/07-reference/05-ai-sdk-errors/ai-invalid-prompt-error.mdx‎

Lines changed: 25 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,25 +9,40 @@ This error occurs when the prompt provided is invalid.
99

1010
## Potential Causes
1111

12-
### UI Messages
12+
### System Messages in `prompt` or `messages`
1313

14-
You are passing a `UIMessage[]` as messages into e.g. `streamText`.
15-
16-
You need to first convert them to a `ModelMessage[]` using `convertToModelMessages()`.
14+
AI SDK functions reject system messages in the `prompt` or `messages` fields to reduce the risk of prompt injection.
15+
Pass system instructions with the `instructions` option instead:
1716

1817
```typescript
19-
import { type UIMessage, generateText, convertToModelMessages } from 'ai';
20-
21-
const messages: UIMessage[] = [
22-
/* ... */
23-
];
18+
import { generateText } from 'ai';
2419

2520
const result = await generateText({
2621
// ...
27-
messages: await convertToModelMessages(messages),
22+
instructions: 'You are a helpful assistant.',
23+
messages,
2824
});
2925
```
3026

27+
If the system messages come from a trusted source, you can allow them with `allowSystemInMessages: true`.
28+
29+
### Empty Messages
30+
31+
The `messages` array (or a `prompt` given as an array) must contain at least one message.
32+
33+
### Provider-Specific Prompt Requirements
34+
35+
Some providers throw this error when a prompt can't be expressed for their API,
36+
for example when an assistant prefix message isn't the final message, or when a part combines options the provider doesn't support.
37+
The error message describes the provider requirement that wasn't met.
38+
39+
<Note>
40+
AI SDK functions do not validate the shape of `ModelMessage` objects at
41+
runtime. If your messages come from an untrusted source, validate them before
42+
passing them to an AI SDK function. See
43+
[`ModelMessage`](/docs/reference/ai-sdk-core/model-message).
44+
</Note>
45+
3146
## Properties
3247

3348
- `prompt`: The invalid prompt value

‎packages/ai/src/batch/batch.test.ts‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -661,18 +661,6 @@ describe('getBatchStatus', () => {
661661
}),
662662
).rejects.toBeInstanceOf(InvalidArgumentError);
663663
});
664-
665-
it('rejects a version 1 batch reference', async () => {
666-
await expect(
667-
getBatchStatus({
668-
provider: createMockBatchApi(),
669-
batch: {
670-
...batchReference,
671-
version: 1,
672-
} as unknown as BatchReference,
673-
}),
674-
).rejects.toBeInstanceOf(InvalidArgumentError);
675-
});
676664
});
677665

678666
describe('getBatchResults', () => {

‎packages/ai/src/batch/batch.ts‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -491,14 +491,6 @@ function validateBatchReference({
491491
batchApi: BatchV4;
492492
batch: BatchReference;
493493
}) {
494-
if (batch.version !== 2) {
495-
throw new InvalidArgumentError({
496-
parameter: 'batch',
497-
value: batch,
498-
message: 'batch must be a supported batch reference',
499-
});
500-
}
501-
502494
if (batch.provider !== batchApi.provider) {
503495
throw new InvalidArgumentError({
504496
parameter: 'provider',

‎packages/ai/src/decide/decide.test.ts‎

Lines changed: 42 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import type { Telemetry } from '../telemetry/telemetry';
1313
import { DecisionMockModelV4 } from '../test/decision-mock-model-v4';
1414
import { experimental_decide as decide } from '../index';
1515
import { VERSION } from '../version';
16+
import type { DecisionStatePart } from './decision-state';
1617

1718
const questions = {
1819
topic: {
@@ -273,74 +274,76 @@ describe('input validation', () => {
273274
const cyclic: Record<string, unknown> = {};
274275
cyclic.self = cyclic;
275276
it.each([
276-
null,
277-
true,
278-
42,
279-
new Date(),
280-
{ value: undefined },
281277
{ value: NaN },
282-
{ fn: () => true },
278+
{ value: Infinity },
283279
cyclic,
284-
[undefined],
285-
new Array(2),
286-
])('rejects non-JSON state %s before I/O', async state => {
280+
{ value: new Array(2) },
281+
[{ type: 'json', value: { bad: NaN } }],
282+
[{ type: 'json', value: cyclic }],
283+
])('rejects non-serializable JSON state %s before I/O', async state => {
287284
const { model, doDecide } = setup();
288285
await expect(
289286
decide({ model, state: state as never, questions }),
290287
).rejects.toBeInstanceOf(InvalidArgumentError);
291288
expect(doDecide).not.toHaveBeenCalled();
292289
});
293290

291+
const outOfOrder: DecisionStatePart[] = [];
292+
outOfOrder[2] = { type: 'text', text: 'third' };
293+
const deleted: DecisionStatePart[] = [
294+
{ type: 'text', text: 'first' },
295+
{ type: 'json', value: {} },
296+
];
297+
delete deleted[0];
298+
it.each([new Array(2), outOfOrder, deleted].map(state => [state]))(
299+
'rejects state parts with array holes %j before I/O',
300+
async state => {
301+
const { model, doDecide } = setup();
302+
await expect(
303+
decide({ model, state: state as never, questions }),
304+
).rejects.toBeInstanceOf(InvalidArgumentError);
305+
expect(doDecide).not.toHaveBeenCalled();
306+
},
307+
);
308+
309+
it('allows repeated JSON references without treating them as cycles', async () => {
310+
const shared = { text: 'hello' };
311+
await expect(
312+
decide({
313+
...setup(),
314+
state: [{ type: 'json', value: [shared, shared] }],
315+
questions,
316+
}),
317+
).resolves.toBeDefined();
318+
});
319+
294320
it.each([
295321
{},
296322
[],
297-
null,
298-
{ test: { type: 'other', instructions: 'test' } },
299323
{ test: { type: 'boolean', instructions: null } },
300-
{ test: { type: 'box' } },
301-
{ test: { type: 'box', instructions: null } },
302-
{ test: { type: 'choice', instructions: 42, criteria: { a: null } } },
303-
{ test: { type: 'score', instructions: false, criteria: ['Low', 'High'] } },
304-
{
305-
test: {
306-
type: 'boolean',
307-
instructions: () => true,
308-
criteria: { true: 'Yes' },
309-
},
310-
},
311-
{ test: { type: 'boolean', criteria: { true: 42 } } },
312324
{ test: { type: 'boolean', criteria: { yes: 'Yes' } } },
313325
{
314326
test: { type: 'boolean', instructions: 'test', criteria: { yes: 'yes' } },
315327
},
316-
{ test: { type: 'choice', instructions: 'test', criteria: {} } },
317-
{ test: { type: 'choice', instructions: 'test', criteria: { a: 42 } } },
318-
{ test: { type: 'score', instructions: 'test', criteria: [] } },
328+
{
329+
test: { type: 'choice', instructions: { n: NaN }, criteria: { a: null } },
330+
},
319331
{
320332
test: {
321-
type: 'score',
333+
type: 'choice',
322334
instructions: 'test',
323-
criteria: ['Low', undefined],
335+
criteria: { a: [Infinity] },
324336
},
325337
},
338+
{ test: { type: 'choice', instructions: 'test', criteria: {} } },
339+
{ test: { type: 'score', instructions: 'test', criteria: [] } },
326340
])('rejects invalid questions %s before I/O', async invalidQuestions => {
327341
const { model, doDecide } = setup();
328342
await expect(
329343
decide({ model, state: 'text', questions: invalidQuestions as never }),
330344
).rejects.toBeInstanceOf(InvalidArgumentError);
331345
expect(doDecide).not.toHaveBeenCalled();
332346
});
333-
334-
it('allows repeated JSON references without treating them as cycles', async () => {
335-
const shared = { text: 'hello' };
336-
await expect(
337-
decide({
338-
...setup(),
339-
state: [{ type: 'json', value: [shared, shared] }],
340-
questions,
341-
}),
342-
).resolves.toBeDefined();
343-
});
344347
});
345348

346349
describe('response validation', () => {
@@ -834,24 +837,6 @@ it('treats an empty array as an empty list of state parts', async () => {
834837
expect(doDecide.mock.calls[0][0].state).toEqual([]);
835838
});
836839

837-
it.each(
838-
[
839-
['legacy JSON array'],
840-
[{ arbitrary: 'object' }],
841-
[{ type: 'image', image: 'iVBORw==' }],
842-
[{ type: 'text', text: 42 }],
843-
[{ type: 'file', mediaType: 'image/png' }],
844-
[{ type: 'json', value: undefined }],
845-
[{ type: 'json', value: { bad: NaN } }],
846-
].map(state => [state]),
847-
)('rejects malformed state parts before I/O: %j', async state => {
848-
const { model, doDecide } = setup();
849-
await expect(
850-
decide({ model, questions, state: state as never }),
851-
).rejects.toBeInstanceOf(InvalidArgumentError);
852-
expect(doDecide).not.toHaveBeenCalled();
853-
});
854-
855840
it('downloads image URLs and sends their bytes to the decision model', async () => {
856841
const { model, doDecide } = setup();
857842
const bytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47]);

0 commit comments

Comments
 (0)