IntermediateUpdated Jan 17, 2026
Testing AI-Powered Features: A Practical Guide
Learn how to write effective tests for AI-powered features, including unit tests, integration tests, and handling non-deterministic outputs.
DP
David Park
Senior Engineer
12 min read
Introduction
Testing AI features presents unique challenges: outputs are non-deterministic and API calls are expensive.
Testing Strategy
- Unit Tests (Many): Business logic, validation
- Integration Tests (Some): API client behavior with mocks
- E2E Tests (Few): Critical user journeys
Unit Testing
typescript
describe('validatePrompt', () => {
it('accepts valid prompts', () => {
expect(validatePrompt('A sunset').valid).toBe(true);
});
it('rejects empty prompts', () => {
const result = validatePrompt('');
expect(result.valid).toBe(false);
expect(result.errors).toContain('Prompt cannot be empty');
});
});Integration Testing with Mocks
typescript
const mockGenerateImage = vi.fn();
vi.mock('./abstrakt-client', () => ({ run: mockGenerateImage }));
describe('ImageService', () => {
it('generates an image', async () => {
mockGenerateImage.mockResolvedValue({
result: { images: [{ url: 'https://example.com/image.png' }] }
});
const result = await service.generate('A sunset');
expect(result.url).toBe('https://example.com/image.png');
});
it('retries on rate limit', async () => {
mockGenerateImage
.mockRejectedValueOnce({ code: 'RATE_LIMITED' })
.mockResolvedValueOnce({ result: { images: [{ url: '...' }] } });
await service.generate('A sunset');
expect(mockGenerateImage).toHaveBeenCalledTimes(2);
});
});Webhook Testing
typescript
describe('Webhook Handler', () => {
it('accepts valid signatures', async () => {
const payload = { event: 'job.completed' };
const signature = signPayload(payload);
const response = await request(app)
.post('/api/webhooks/abstrakt')
.set('x-abstrakt-signature', signature)
.send(payload);
expect(response.status).toBe(200);
});
});E2E Testing
typescript
test('generates an image', async ({ page }) => {
await page.goto('/generate');
await page.fill('[data-testid="prompt"]', 'A sunset');
await page.click('[data-testid="generate"]');
await expect(page.locator('[data-testid="image"]')).toBeVisible({
timeout: 60000
});
});CI/CD
yaml
jobs:
test:
steps:
- run: npm run test:unit
- run: npm run test:integration
env:
ABSTRAKT_API_KEY: ${{ secrets.TEST_KEY }}#testing#jest#vitest#mocking#qa