+1 (415) 779-8456
StaticCraft

Cutting Gatsby Build Times: CI Caching, Query Discipline, and DSG

A Gatsby build that took four minutes when the site had 300 pages takes forty when it has 8,000 — and the slowdown is rarely linear or evenly distributed. Deploys stop being cheap, preview builds stop being useful, and the team starts batching content changes because nobody wants to wait. This is the single most common non-bug complaint we get about mature Gatsby 5 projects.

The fix is almost never "buy a bigger CI runner". It is finding out which of the four build phases is actually slow, then applying the specific remedy for that phase. This tutorial walks through the measurement first, then the remedies in the order that usually pays best.

The four phases, and what each one means

Run a build with timing detail:

GATSBY_CPU_COUNT=logical_cores npx gatsby build --verbose 2>&1 | tee build.log

Gatsby prints an activity timer for each step. Group them mentally into four phases:

  1. Sourcing — source and transform nodes. Fetching from your CMS, reading the filesystem, running transformer plugins.
  2. Schema and queries — building schema, run static queries, run page queries. Every page's GraphQL query executed against the node store.
  3. Asset processing — Processing images, write out requires. Almost always sharp.
  4. Bundling and HTML — building JavaScript bundles (webpack), Building static HTML for pages (SSR of every page).

Pull the numbers out of the log so you are arguing about data:

grep -E "^success|^warn" build.log | sort -t'-' -k2 -rn | head -20

In our experience the distribution on a slow content site is roughly: images 40%, page queries 25%, static HTML 20%, webpack 10%, sourcing 5%. On a slow app-like site it inverts: webpack and HTML dominate. Which one you have determines everything below, so do not skip this step.

Fix 1: stop reprocessing images you already processed

sharp is usually the biggest single line item, and it is also the most cacheable. Gatsby stores processed derivatives in .cache and public/static. If your CI starts from an empty checkout every run, you are paying full image cost on every deploy, forever.

Restore both directories in CI. GitHub Actions:

- uses: actions/cache@v4
  with:
    path: |
      .cache
      public
    key: gatsby-${{ github.ref_name }}-${{ hashFiles('package-lock.json') }}-${{ github.sha }}
    restore-keys: |
      gatsby-${{ github.ref_name }}-${{ hashFiles('package-lock.json') }}-
      gatsby-${{ github.ref_name }}-

Two details people get wrong:

  • Include package-lock.json in the key. Gatsby invalidates its cache when plugin versions change; a stale cache across a dependency bump produces confusing "cache is invalid, deleting" runs that cost more than they save.
  • Cache public too, not just .cache. Image derivatives live under public/static. Caching .cache alone still forces sharp to re-emit files.

Netlify users: install netlify-plugin-gatsby (or the official Gatsby build plugin) which does this persistence for you. Vercel does it automatically for .cache if the framework preset is detected — verify in the build log that you see Restored build cache.

Then reduce the work itself. Two changes with large effects:

// gatsby-config.js
plugins: [
  {
    resolve: 'gatsby-plugin-sharp',
    options: {
      defaults: {
        formats: ['auto', 'webp'],   // drop 'avif' unless you measured the win
        placeholder: 'dominantColor', // blurred placeholders are the expensive ones
        breakpoints: [750, 1080, 1366, 1920],
      },
    },
  },
]

avif encoding is several times slower than webp for a few percent of file size on typical photography. placeholder: 'blurred' generates and base64-encodes an extra tiny image per source image; dominantColor is close to free and looks fine for most layouts. Trimming the default breakpoint list from six sizes to four removes a third of the encoding work outright.

Finally, check that you are not sourcing images you never render. A gatsby-source-filesystem pointed at a folder containing 4,000 originals when the site uses 200 of them will still run the transformer over all of them.

Fix 2: make page queries cheap and few

run page queries scales with the number of pages times the cost of each query. Both factors are usually fixable.

Do not query fields you do not render. The single worst offender is pulling full body/html for every item in a listing query. A blog index that renders titles and dates but queries html forces the transformer to render every post's HTML to satisfy the index page.

# bad: index page pulls every post body
{ allMarkdownRemark { nodes { html frontmatter { title } } } }

# good
{ allMarkdownRemark { nodes { excerpt(pruneLength: 160) frontmatter { title date } } } }

Do not query image data for images that are off-screen and paginated away. gatsby-plugin-image's GatsbyImage fragments are not free at query time either.

Watch useStaticQuery in shared components. A static query in a header component runs once, which is fine. A static query inside a component rendered per-item is a common accident that turns into thousands of resolver calls.

After trimming, re-run and compare the run page queries line. On sites where the index and tag pages were pulling full bodies, we routinely see this phase drop by 60–80%.

Fix 3: stop building HTML for pages nobody visits

This is the lever most teams have not pulled. Gatsby 5 supports Deferred Static Generation (DSG): the page is defined at build time but its HTML is rendered on first request, then cached like a static file.

Archive pages, paginated listings beyond page 3, old posts, per-tag pages — these are typically 80% of a large site's page count and 2% of its traffic. Defer them:

// gatsby-node.js
exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions;
  const { data } = await graphql(`{ allMarkdownRemark(sort: {frontmatter: {date: DESC}}) { nodes { id fields { slug } frontmatter { date } } } }`);

  const cutoff = new Date();
  cutoff.setFullYear(cutoff.getFullYear() - 1);

  data.allMarkdownRemark.nodes.forEach((node, i) => {
    createPage({
      path: node.fields.slug,
      component: require.resolve('./src/templates/post.js'),
      context: { id: node.id },
      // recent posts build eagerly; the long tail is deferred
      defer: new Date(node.frontmatter.date) < cutoff && i > 50,
    });
  });
};

Caveats worth knowing before you ship this:

  • DSG needs a runtime. It works on Netlify (via the adapter), Vercel, and self-hosted gatsby serve. On a pure S3/CloudFront or GitHub Pages deploy there is nothing to render deferred pages, and defer silently gets you nothing useful. Check your adapter first — Gatsby 5's adapters replaced the old per-host plugins.
  • First request pays the cost. Keep anything you care about ranking or converting on out of the deferred set, or accept a cold-start on the crawler's first hit.
  • Deferred pages do not appear in public/ at build time. Any post-build script that walks the output directory — link checkers, axe sweeps, sitemap generators that read the filesystem — needs to be aware of that.

Measured effect: a 9,000-page documentation site we worked on deferred 7,400 pages and cut Building static HTML from 14 minutes to under 2.

Fix 4: give webpack less to do

If your bottleneck is building JavaScript bundles, the usual causes are a moment.js-scale dependency pulled in globally, a barrel-file import that defeats tree shaking, or source maps you do not use.

// gatsby-node.js
exports.onCreateWebpackConfig = ({ stage, actions }) => {
  if (stage === 'build-javascript') {
    actions.setWebpackConfig({ devtool: false }); // if you don't upload maps
  }
};

Then look at what is actually in the bundle:

npm i -D gatsby-plugin-webpack-bundle-analyser-v2

Import a date library's single function instead of the whole package, replace an icon set import with per-icon imports, and lazy-load anything below the fold that ships its own runtime (charts, maps, editors). This phase also parallelises poorly, so it is the one place where a faster CPU on the runner genuinely helps.

Fix 5: incremental sourcing from your CMS

If source and transform nodes is your slow phase, you are probably refetching the entire content set every build. Most modern source plugins support delta sourcing — gatsby-source-contentful with enableTags/sync tokens, gatsby-source-drupal with the webhook module, gatsby-source-wordpress with its built-in incremental fetch. All of them depend on .cache surviving between builds, which is Fix 1 again.

Two practical rules: set a webhook that triggers builds on publish rather than polling on a cron, and make sure preview builds use the same warm cache as production builds rather than a separate cold one.

Put a ceiling on it in CI

Once the build is fast, keep it fast. Fail the pipeline if it regresses:

START=$(date +%s)
npx gatsby build
ELAPSED=$(( $(date +%s) - START ))
echo "build_seconds=$ELAPSED"
if [ "$ELAPSED" -gt 600 ]; then
  echo "::warning::Build took ${ELAPSED}s (budget 600s)"
fi

Log the number on every run and put it on a chart. Build time regresses the way page weight does — quietly, one plugin at a time — and it is much easier to find the culprit in the commit that added 90 seconds than in the six months since anyone looked.

The order we work in

  1. Measure. Get the per-phase breakdown before changing anything.
  2. Fix caching in CI. Cheapest possible win, no code changes.
  3. Trim image settings and unused sourced files.
  4. Trim page queries.
  5. Defer the long tail with DSG, if your host supports it.
  6. Attack webpack only if the log says webpack is the problem.

Most sites we take on land between a 3x and 8x improvement without touching a line of component code. If your Gatsby build has crept past the point where anyone wants to deploy, or you want a second opinion on whether the build is worth optimising versus migrating, get in touch — we do fixed-scope build audits that start with the same log analysis above.