Most of the Gatsby sites we are called into are not under active development. They were built two to four years ago, they still serve real traffic, and someone touches them a handful of times a year: a content model tweak, a Node upgrade, a security patch on a transitive dependency. That pattern is exactly the one automated tests are best at protecting, and it is also the pattern where test suites are usually missing entirely.
The good news is that a static site is the easiest thing in the world to test. There is no server state, no login-only rendering path, and the artifact under test — the public/ directory — is deterministic. You do not need a 400-test suite. You need a harness that fails loudly when a build silently degrades.
This tutorial builds that harness in four layers, from cheapest to most expensive:
- Assertions on the build output itself (seconds, no browser).
- A Playwright smoke suite over the served static files.
- Link, redirect, and metadata checks.
- Visual regression gates that do not flake.
Everything here targets Gatsby 5 on Node 20/22, and it assumes CI on GitHub Actions — but nothing is Actions-specific.
Layer 0: make the build fail instead of degrade
Before adding tests, remove the ways Gatsby hides breakage. Two settings matter.
First, make GraphQL and page-creation warnings fatal in CI. A missing field in a query usually renders as an empty string instead of an error, which is how sites lose their entire <title> set without anyone noticing.
// gatsby-node.js
exports.onPostBuild = async ({ reporter, graphql }) => {
const { data, errors } = await graphql(`
{
allSitePage { totalCount }
allMarkdownRemark(filter: { frontmatter: { title: { eq: null } } }) {
nodes { fileAbsolutePath }
}
}
`);
if (errors) throw errors;
const missing = data.allMarkdownRemark.nodes;
if (missing.length) {
reporter.panic(
`Content missing a title:\n${missing.map((n) => n.fileAbsolutePath).join('\n')}`
);
}
reporter.info(`Built ${data.allSitePage.totalCount} pages`);
};
Second, pin a page-count floor. The single most common silent Gatsby failure is a source plugin that authenticates, gets a 401 or an empty response, and cheerfully builds a site with a homepage and nothing else. A floor turns that into a red build:
// gatsby-node.js (continued inside onPostBuild)
const MIN_PAGES = Number(process.env.MIN_PAGES || 0);
if (MIN_PAGES && data.allSitePage.totalCount < MIN_PAGES) {
reporter.panic(
`Only ${data.allSitePage.totalCount} pages built, expected at least ${MIN_PAGES}. ` +
`A source plugin probably returned nothing.`
);
}
Set MIN_PAGES in CI to roughly 90% of your current page count and bump it when the site grows. This one check has caught more real incidents on our client projects than every browser test combined.
Layer 1: assert the artifact, not the app
These tests read files in public/ after a build. They run in about a second and need no browser. Plain Node's built-in test runner is enough:
// tests/build.test.js
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync, existsSync } from 'node:fs';
import { globSync } from 'node:fs';
const read = (p) => readFileSync(`public/${p}`, 'utf8');
test('critical routes exist', () => {
for (const route of ['index.html', 'services/index.html', 'contact/index.html']) {
assert.ok(existsSync(`public/${route}`), `missing ${route}`);
}
});
test('homepage ships real HTML, not an empty shell', () => {
const html = read('index.html');
assert.ok(html.length > 5000, 'homepage suspiciously small');
assert.match(html, /<title>[^<]{10,}<\/title>/);
assert.match(html, /<meta name="description" content="[^"]{50,}"/);
});
test('no placeholder or template leakage', () => {
const html = read('index.html');
for (const bad of ['lorem ipsum', 'undefined', '{{', 'TODO:']) {
assert.ok(!html.toLowerCase().includes(bad.toLowerCase()), `found "${bad}"`);
}
});
test('sitemap and robots exist and agree', () => {
assert.ok(existsSync('public/robots.txt'));
const sitemap = read('sitemap-index.xml');
assert.match(sitemap, /<loc>https:\/\//);
});
The undefined check looks crude. It is also the fastest way to catch ${post.frontmatter.subtitle} rendering as the literal string undefined across 200 pages.
A useful sibling check is the hydration-parity smell test: if a page's rendered body is under a few hundred bytes while its JavaScript bundle is large, something that should be static is being rendered client-side. Assistants and non-JS crawlers see the small version.
Layer 2: a Playwright smoke suite over the static output
Run Playwright against the built files, not against gatsby develop. Development mode uses a different bundler path, a different data layer, and will happily pass while production is broken.
npm i -D @playwright/test
npx playwright install --with-deps chromium
// playwright.config.js
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests/e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 1 : 0,
reporter: process.env.CI ? [['github'], ['html', { open: 'never' }]] : 'list',
use: {
baseURL: 'http://127.0.0.1:9000',
trace: 'on-first-retry',
},
webServer: {
command: 'npx gatsby serve --port 9000',
url: 'http://127.0.0.1:9000',
reuseExistingServer: !process.env.CI,
timeout: 120_000,
},
projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
});
gatsby serve mirrors production routing closely enough for smoke coverage, including trailing-slash behaviour and 404 handling. Where your host does redirects at the edge (Netlify _redirects, Cloudflare rules), those are not covered here — test them separately against a deploy preview, which we come back to below.
Now the suite. Keep it small and behavioural:
// tests/e2e/smoke.spec.js
import { test, expect } from '@playwright/test';
const ROUTES = ['/', '/services/', '/tutorials/', '/contact/'];
test.describe('smoke', () => {
for (const path of ROUTES) {
test(`${path} renders and has no console errors`, async ({ page }) => {
const errors = [];
page.on('pageerror', (e) => errors.push(e.message));
page.on('console', (m) => m.type() === 'error' && errors.push(m.text()));
const res = await page.goto(path);
expect(res.status()).toBe(200);
await expect(page.locator('h1')).toBeVisible();
await expect(page).toHaveTitle(/\w{5,}/);
expect(errors, `console errors on ${path}`).toEqual([]);
});
}
test('client-side navigation works after hydration', async ({ page }) => {
await page.goto('/');
await page.getByRole('link', { name: /services/i }).first().click();
await expect(page).toHaveURL(/\/services\/?$/);
await expect(page.locator('h1')).toBeVisible();
});
test('unknown route serves the 404 page', async ({ page }) => {
const res = await page.goto('/definitely-not-a-real-page/');
expect(res.status()).toBe(404);
await expect(page.getByText(/not found/i)).toBeVisible();
});
test('contact form posts and confirms', async ({ page }) => {
await page.route('**/api/**', (route) =>
route.fulfill({ status: 200, body: '{"ok":true}' })
);
await page.goto('/contact/');
await page.getByLabel(/name/i).fill('Test Person');
await page.getByLabel(/email/i).fill('test@example.com');
await page.getByLabel(/message/i).fill('Gatsby migration enquiry');
await page.getByRole('button', { name: /send|submit/i }).click();
await expect(page.getByText(/thank|received/i)).toBeVisible();
});
});
The console-error assertion is the highest-value line in that file. Gatsby hydration mismatches, a dead third-party script, and a plugin that throws on a specific route all surface as console errors long before anyone reports a visual bug. If your site loads noisy third-party tags, filter by pattern rather than dropping the check:
const IGNORE = [/googletagmanager/, /ERR_BLOCKED_BY_CLIENT/];
const noisy = (t) => IGNORE.some((re) => re.test(t));
The form test matters because on post-Gatsby-Cloud sites the form endpoint has usually been rewired to Netlify Forms, a serverless function, or a third-party service — and nobody notices when it breaks. Route-mocking keeps the test hermetic; pair it with a monthly real submission check against production.
Layer 3: links, redirects, and metadata
Internal 404s accumulate quietly on content sites. Crawl the built output rather than the live site so the check runs pre-merge:
npm i -D linkinator
npx linkinator public --recurse --silent \
--skip "^https?://(?!localhost)" \
--format json > link-report.json
Skipping external URLs keeps CI deterministic; run a full external crawl weekly on a schedule instead, where a flaky third-party 503 does not block a deploy.
Redirects deserve their own test, especially if you have migrated URLs. Keep the expected mapping in a data file that both gatsby-node.js and the test read:
// redirects.js
module.exports = [
{ from: '/old-consulting/', to: '/gatsby-consulting/' },
{ from: '/blog/', to: '/tutorials/' },
];
// tests/e2e/redirects.spec.js
import { test, expect } from '@playwright/test';
import redirects from '../../redirects.js';
const BASE = process.env.DEPLOY_URL; // deploy preview URL
test.describe('redirects', () => {
test.skip(!BASE, 'no DEPLOY_URL set');
for (const { from, to } of redirects) {
test(`${from} -> ${to}`, async ({ request }) => {
const res = await request.get(BASE + from, { maxRedirects: 0 });
expect([301, 308]).toContain(res.status());
expect(res.headers()['location']).toContain(to);
});
}
});
Because host-level redirects only exist once deployed, this spec runs against the deploy preview URL, not gatsby serve. Netlify exposes it as DEPLOY_PRIME_URL; Cloudflare Pages and Vercel have equivalents.
Finally, guard the metadata that drives search and AI-assistant visibility — canonical tags, JSON-LD validity, and og:image presence:
test('article pages carry valid JSON-LD', async ({ page }) => {
await page.goto('/tutorials/gatsby-static-search-pagefind/');
const blocks = await page.locator('script[type="application/ld+json"]').allTextContents();
expect(blocks.length).toBeGreaterThan(0);
for (const raw of blocks) {
const parsed = JSON.parse(raw); // throws on malformed JSON-LD
expect(parsed['@context']).toMatch(/schema\.org/);
}
await expect(page.locator('link[rel="canonical"]')).toHaveCount(1);
});
While you are here, add an accessibility gate with @axe-core/playwright on three or four representative templates. It costs a few seconds and prevents the slow regression of contrast and landmark structure.
Layer 4: visual regression that does not flake
Visual diffing has a bad reputation, entirely earned by suites that screenshot whole pages containing carousels, fonts loading at different moments, and dates. Three rules make it usable:
Rule 1: screenshot components and sections, not full pages, except for one or two key templates.
Rule 2: freeze everything non-deterministic.
// tests/e2e/visual.spec.js
import { test, expect } from '@playwright/test';
test.beforeEach(async ({ page }) => {
await page.emulateMedia({ reducedMotion: 'reduce' });
await page.addStyleTag({
content: `*, *::before, *::after { animation: none !important; transition: none !important; }
.js-timestamp, [data-testid="relative-date"] { visibility: hidden !important; }`,
});
await page.route(/googletagmanager|hotjar|intercom/, (r) => r.abort());
});
test('homepage hero is visually stable', async ({ page }) => {
await page.goto('/');
await page.evaluate(() => document.fonts.ready);
await expect(page.locator('[data-testid="hero"]')).toHaveScreenshot('hero.png', {
maxDiffPixelRatio: 0.01,
});
});
Rule 3: generate baselines inside the same container CI uses. Font rasterisation differs between macOS and Linux, so baselines produced on a laptop will fail forever in CI. Update them with a container run:
docker run --rm -v "$(pwd):/work" -w /work \
mcr.microsoft.com/playwright:v1.49.0-jammy \
npx playwright test visual --update-snapshots
Pin the Playwright container tag to the same version as your dependency, and bump both together — a browser upgrade will legitimately change a few pixels and you want that in a separate, reviewable commit.
Wiring it into CI
# .github/workflows/ci.yml
name: CI
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- name: Restore Gatsby cache
uses: actions/cache@v4
with:
path: |
.cache
public
key: gatsby-${{ hashFiles('package-lock.json') }}-${{ github.sha }}
restore-keys: gatsby-${{ hashFiles('package-lock.json') }}-
- run: npx gatsby build
env:
MIN_PAGES: 180
CI: true
- run: node --test tests/build.test.js
- run: npx linkinator public --recurse --silent --skip "^https?://(?!localhost)"
- uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: pw-${{ hashFiles('package-lock.json') }}
- run: npx playwright install --with-deps chromium
- run: npx playwright test
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/
retention-days: 7
Two details worth copying. Reusing the Gatsby .cache and public directories keeps the build step honest about incremental behaviour and cuts CI time substantially on large sites — the same caching discipline we cover in our build-times tutorial. And uploading the Playwright report only on failure means the artifact you need is always there and never costs storage on green runs.
Add a scheduled run for the parts that legitimately depend on the outside world:
on:
schedule:
- cron: '0 6 * * 1'
Weekly, run the external link crawl, a real (not mocked) form submission against a staging endpoint, and a Lighthouse or CrUX check. Failures there are informational, not merge-blocking.
What we deliberately do not test
A harness earns its keep only if it stays green for the right reasons. On low-change Gatsby sites we skip unit tests for presentational components, snapshot tests of rendered React trees, and cross-browser matrices. The failure modes that actually hit these projects are: a source plugin returning nothing, a dependency upgrade breaking hydration, a form endpoint going dark, and URLs shifting during a migration. Every layer above targets one of those directly.
Budget roughly a day to install all four layers on an existing site, and expect the page-count floor and console-error assertions to pay for it within the first two dependency bumps.
Getting help
We install this harness as part of most Gatsby maintenance and migration engagements, and we will happily set it up as a standalone piece of work — including the CI wiring and a baseline set of visual snapshots generated in your own runner image. If you have a Gatsby site that nobody wants to touch because nobody knows what will break, get in touch and we will scope it.