+1 (415) 779-8456
StaticCraft

Don't Lose the URLs: Building a Redirect Map for a Gatsby Replatform

Every Gatsby replatform — to Astro, to Next.js, or just a rebuild in Gatsby 5 — has one failure mode that costs real money and shows up six weeks later: the URLs moved. Not all of them, usually. A trailing slash disappeared, /blog/ became /insights/, pagination changed shape, image files got new hashes, and the site that used to rank for forty phrases now ranks for nine. Nobody notices during QA because QA clicks links inside the new site, and inside the new site every link is correct.

This tutorial is the checklist we run on every migration: capture the old URL surface before you cut over, build a redirect map from it, ship the redirects at the edge, and verify with a crawl instead of a vibe.

1. Capture the old URL surface before anything moves

You need three lists, and you need them while the old site is still live.

Everything the old build emits. If you still have the Gatsby repo, the most complete source is public/ after a build — no crawler misses a page that way:

npx gatsby build
cd public && find . -name 'index.html' \
  | sed 's|^\.||; s|/index\.html$|/|' \
  | sort > ../urls-built.txt
wc -l ../urls-built.txt

Add the non-HTML assets that are linked from elsewhere — PDFs, feeds, downloads:

cd public && find . -type f \
  \( -name '*.pdf' -o -name '*.xml' -o -name '*.zip' -o -name '*.csv' \) \
  | sed 's|^\.||' | sort >> ../urls-built.txt

Everything search engines know about. The built list and the indexed list are never the same. Old sites accumulate URLs that no longer exist in the repo but still have links and rankings pointing at them. Export from Google Search Console (Pages report → export, or the URL inspection API for a bigger site) and from Bing Webmaster Tools. Pull the sitemap too:

curl -s https://www.example.com/sitemap-index.xml \
  | grep -o '<loc>[^<]*</loc>' | sed 's|</\?loc>||g' > sitemaps.txt
while read s; do curl -s "$s" | grep -o '<loc>[^<]*</loc>' | sed 's|</\?loc>||g'; done < sitemaps.txt \
  | sort -u > urls-sitemap.txt

Everything that actually gets traffic or links. Pull the last 12 months of landing pages from analytics (12, not 3 — seasonal pages are exactly the ones you will forget), and the top linked pages from whatever backlink tool you have. These are the URLs where a mistake is expensive; the rest are cleanup.

Merge and dedupe:

cat urls-built.txt urls-sitemap.txt urls-gsc.txt urls-analytics.txt \
  | sed 's|^https\?://[^/]*||' | sed 's|?.*$||' \
  | sort -u > urls-old.txt

Keep urls-old.txt in the repo. It is the contract the new site has to honour.

2. Settle the trailing-slash question once

This is the single most common migration regression, and it is boring enough that everyone skips it.

Gatsby 4 and 5 default to trailingSlash: 'always' — gatsby-config.js accepts 'always', 'never', or 'ignore', and the setting controls both the emitted file layout and the paths in your sitemap. Astro has trailingSlash in astro.config.mjs; Next.js has trailingSlash in next.config.js, defaulting to false. Two different frameworks, two different defaults, and hosts add their own normalisation on top.

Decide which form is canonical, then make three things agree: the framework config, the host's redirect behaviour, and every <link rel="canonical"> you emit. A quick check against the live old site tells you what it does today:

for u in / /services/ /services /blog/some-post/ /blog/some-post; do
  printf '%-24s %s\n' "$u" "$(curl -s -o /dev/null -w '%{http_code} -> %{redirect_url}' "https://www.example.com$u")"
done

If the old site served both forms with a 200, you have duplicate content that the migration is a good chance to fix — pick one, 301 the other, and expect a brief ranking wobble that settles. If the old site 301'd one to the other, replicate that direction exactly. Flipping it mid-migration doubles the number of hops and is the fastest way to create a redirect chain.

3. Generate the redirect map, do not hand-write it

For most migrations, 80–90% of URLs map by rule and the rest are one-offs. Write the rules as code so you can re-run them when the content team renames one more thing.

// scripts/build-redirect-map.mjs
import fs from 'node:fs';

const oldUrls = fs.readFileSync('urls-old.txt', 'utf8').split('\n').filter(Boolean);
const newUrls = new Set(
  fs.readFileSync('urls-new.txt', 'utf8').split('\n').filter(Boolean),
);

// Ordered rules: first match wins.
const rules = [
  [/^\/blog\/(.+)$/, '/insights/$1'],
  [/^\/blog\/page\/(\d+)\/$/, '/insights/page/$1/'],
  [/^\/author\/[^/]+\/?$/, '/insights/'],
  [/^\/tags\/([^/]+)\/?$/, '/topics/$1/'],
];

// Hand-curated exceptions that no rule can express.
const manual = JSON.parse(fs.readFileSync('redirects-manual.json', 'utf8'));

const out = [];
const unresolved = [];

for (const url of oldUrls) {
  if (newUrls.has(url)) continue;            // URL survived, no redirect needed
  if (manual[url]) { out.push([url, manual[url]]); continue; }

  const rule = rules.find(([re]) => re.test(url));
  const target = rule ? url.replace(rule[0], rule[1]) : null;

  if (target && newUrls.has(target)) out.push([url, target]);
  else unresolved.push(url);
}

fs.writeFileSync('redirects.json', JSON.stringify(Object.fromEntries(out), null, 2));
fs.writeFileSync('unresolved.txt', unresolved.join('\n'));
console.log(`mapped ${out.length}, unresolved ${unresolved.length}`);

Note what the script refuses to do: it never emits a redirect to a URL that does not exist in urls-new.txt. A redirect to a 404 is worse than a 404, because it hides the problem from your own monitoring.

The unresolved.txt file is the actual deliverable of this step. Work it down by hand with the content owner. For each one the options are: build the page, redirect to the closest genuine equivalent, or let it 404 (or better, 410) on purpose. "Redirect everything to the homepage" is not a fourth option — search engines treat a mass homepage redirect as a soft 404 and it wipes the equity you were trying to keep, plus it strands the human who clicked.

4. Ship redirects at the edge, not in the app

Gatsby's createRedirect in gatsby-node.js is only half a mechanism: it writes host-specific config through an adapter (Netlify's _redirects, for instance) and otherwise falls back to a client-side meta-refresh page, which is a 200 with a JavaScript hop — not a 301. For migration-scale redirects, generate the host's native config from redirects.json and let the CDN answer.

Netlify (static/_redirects, or emitted in onPostBuild):

// gatsby-node.js
import fs from 'node:fs/promises';
export const onPostBuild = async () => {
  const map = JSON.parse(await fs.readFile('redirects.json', 'utf8'));
  const lines = Object.entries(map).map(([from, to]) => `${from}  ${to}  301!`);
  await fs.appendFile('public/_redirects', lines.join('\n') + '\n');
};

The ! force flag matters: without it, Netlify skips the redirect when a real file exists at that path, which is exactly the case during a partial migration.

Cloudflare Pages uses the same syntax in public/_redirects but caps the file at 2,100 rules — past that, move to a Worker with the map in KV. Vercel wants redirects in vercel.json, and next.config.js redirects() is fine for a few hundred entries but is evaluated per request, so generate vercel.json for large maps. On S3 + CloudFront, a CloudFront Function doing a lookup against an inlined map is the cheap version; anything over a few hundred KB of rules belongs in a KeyValueStore.

Three rules regardless of host:

  • 301, not 302. Use 302 only if you genuinely plan to reverse it.
  • No chains. If /a/ → /b/ and later /b/ → /c/, rewrite the first rule to point at /c/. Flatten the map before you ship it; the script can do this in a loop.
  • Preserve the query string where it means something (campaign tags, paginated filters). Most hosts do by default; verify rather than assume.

5. Verify with a crawl, then again after cutover

Run this against a preview deploy of the new site before DNS moves:

# every old URL must answer 200 or 301-to-200
BASE=https://preview-new-site.example.dev
fail=0
while read u; do
  read -r code loc < <(curl -s -o /dev/null -w '%{http_code} %{redirect_url}' "$BASE$u" | tr -d '\r')
  if [ "$code" = "301" ]; then
    final=$(curl -s -o /dev/null -w '%{http_code}' -L "$BASE$u")
    hops=$(curl -s -o /dev/null -w '%{num_redirects}' -L "$BASE$u")
    [ "$final" = "200" ] && [ "$hops" -le 1 ] || { echo "CHAIN/BROKEN $u -> $loc ($final, $hops hops)"; fail=1; }
  elif [ "$code" != "200" ]; then
    echo "MISSING $u ($code)"; fail=1
  fi
done < urls-old.txt
exit $fail

Wire that into CI as a release gate on the migration branch. It is a few minutes of runtime for a few thousand URLs and it catches the regression class that costs the most.

After cutover, the follow-ups that matter:

  • Submit the new sitemap in Search Console and leave the old sitemap available for a few weeks if the URLs changed — it gives crawlers a list of pages to re-discover and re-map.
  • Watch Search Console's Pages report for a rising "Not found (404)" or "Page with redirect" count. A spike in the first is your unresolved list coming back to haunt you.
  • Check server/CDN logs for 404s with a referrer — those are real humans hitting real broken links, and they are worth fixing the same day.
  • Keep the redirect map forever. Deleting migration redirects "because it's been a year" is how sites lose links twice.

What good looks like

Traffic dips for one to three weeks after a replatform even when the mapping is perfect — crawlers have to re-process the whole site. What should not happen is a dip that does not recover, or a steady trickle of 404s in the logs. If you are eight weeks out and impressions have not returned to the old baseline, the problem is usually in unresolved.txt: pages that were quietly dropped because nobody could say what they were for.

Do the capture step first. Everything else in this list is mechanical once you have an honest inventory of what the old site actually served.

If you are planning a Gatsby migration and want the URL mapping handled by someone who has done it before — or a second opinion on a migration that already lost traffic — get in touch.