Skip to content

Conversation

@jaffrepaul
Copy link
Contributor

@jaffrepaul jaffrepaul commented Nov 10, 2025

DESCRIBE YOUR PR

The 404 link checker GitHub Action was processing over 9,000 pages despite the repository containing only ~2,000 source files. The Sentry docs use a "common files" architecture where a single source file generates multiple URLs across platforms and guides and we were checking every generated page independently.

Also optimized the cache strategy and made moves to what's bundled during the build to avoid Vercel's 250MB limit on serverless function size.

Implemented source-based deduplication using the existing /api/source-map endpoint:

  • Fetch source-map that maps each slug to its source file
  • Track which source files have been checked
  • Only check one page per unique source file
  • Always check API-generated pages (no source file)

That gets us...

  • ~72% faster - Checks ~2,500 unique pages instead of 9,000
  • ~72% fewer HTTP requests - Massive reduction in server load
  • Same coverage - Still validates all pages, just efficiently

IS YOUR CHANGE URGENT?

Help us prioritize incoming PRs by letting us know when the change needs to go live.

  • Urgent deadline (GA date, etc.):
  • Other deadline:
  • None: Not urgent, can wait up to 1 week+

SLA

  • Teamwork makes the dream work, so please add a reviewer to your PRs.
  • Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it.
    Thanks in advance for your help!

PRE-MERGE CHECKLIST

Make sure you've checked the following before merging your changes:

  • Checked Vercel preview for correctness, including links
  • PR was reviewed and approved by any necessary SMEs (subject matter experts)
  • PR was reviewed and approved by a member of the Sentry docs team

LEGAL BOILERPLATE

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

EXTRA RESOURCES

@vercel
Copy link

vercel bot commented Nov 10, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
develop-docs Ready Ready Preview Comment Nov 10, 2025 9:53pm
sentry-docs Ready Ready Preview Comment Nov 10, 2025 9:53pm

@codecov
Copy link

codecov bot commented Nov 10, 2025

Bundle Report

Changes will decrease total bundle size by 2.68MB (-11.54%) ⬇️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
sentry-docs-client-array-push 10.44MB 269.19kB (2.65%) ⬆️
sentry-docs-server-cjs 9.67MB -2.95MB (-23.36%) ⬇️

Affected Assets, Files, and Routes:

view changes for bundle: sentry-docs-server-cjs

Assets Changed:

Asset Name Size Change Total Size Change (%)
729.js (New) 1.74MB 1.74MB 100.0% 🚀
../instrumentation.js -10 bytes 1.07MB -0.0%
143.js (New) 1.06MB 1.06MB 100.0% 🚀
523.js 1.04MB 1.04MB 88134.12% ⚠️
../app/[[...path]]/page.js.nft.json -1.99kB 765.34kB -0.26%
../app/platform-redirect/page.js.nft.json -1.99kB 765.25kB -0.26%
../app/[[...path]]/page.js 62 bytes 619.92kB 0.01%
../app/api/source-map/route.js.nft.json (New) 262.13kB 262.13kB 100.0% 🚀
../app/sitemap.xml/route.js.nft.json -512.56kB 251.91kB -67.05%
64.js (New) 196.79kB 196.79kB 100.0% 🚀
151.js (New) 121.82kB 121.82kB 100.0% 🚀
236.js (New) 120.2kB 120.2kB 100.0% 🚀
548.js (New) 67.9kB 67.9kB 100.0% 🚀
../app/platform-redirect/page.js 60 bytes 32.28kB 0.19%
153.js (New) 30.43kB 30.43kB 100.0% 🚀
164.js (New) 24.84kB 24.84kB 100.0% 🚀
../app/api/source-map/route.js (New) 9.48kB 9.48kB 100.0% 🚀
../app/sitemap.xml/route.js -25.38kB 8.9kB -74.04%
../../types/app/api/source-map/route.ts (New) 8.72kB 8.72kB 100.0% 🚀
../app/_not-found/page.js 1.94kB 6.52kB 42.37% ⚠️
../app/api/ip-ranges/route.js -2 bytes 5.79kB -0.03%
../app/robots.txt/route.js -4 bytes 5.02kB -0.08%
../pages/_error.js -4 bytes 4.28kB -0.09%
../webpack-runtime.js -2 bytes 3.28kB -0.06%
597.js (New) 1.36kB 1.36kB 100.0% 🚀
556.js (New) 1.36kB 1.36kB 100.0% 🚀
672.js (New) 1.36kB 1.36kB 100.0% 🚀
180.js (New) 1.27kB 1.27kB 100.0% 🚀
247.js (New) 1.27kB 1.27kB 100.0% 🚀
11.js (New) 1.26kB 1.26kB 100.0% 🚀
980.js (New) 1.25kB 1.25kB 100.0% 🚀
51.js (New) 1.25kB 1.25kB 100.0% 🚀
../pages/_document.js -3 bytes 1.2kB -0.25%
129.js (New) 1.1kB 1.1kB 100.0% 🚀
160.js (New) 1.1kB 1.1kB 100.0% 🚀
468.js (New) 1.1kB 1.1kB 100.0% 🚀
445.js (New) 968 bytes 968 bytes 100.0% 🚀
202.js (New) 968 bytes 968 bytes 100.0% 🚀
97.js (New) 965 bytes 965 bytes 100.0% 🚀
../app/_not-found/page.js.nft.json -32 bytes 824 bytes -3.74%
../app/api/ip-ranges/route.js.nft.json -12 bytes 597 bytes -1.97%
../app/robots.txt/route.js.nft.json -12 bytes 459 bytes -2.55%
../instrumentation.js.nft.json -10 bytes 274 bytes -3.52%
../pages/_document.js.nft.json -2 bytes 213 bytes -0.93%
../pages/_error.js.nft.json -2 bytes 157 bytes -1.26%
1729.js (Deleted) -1.74MB 0 bytes -100.0% 🗑️
1614.js (Deleted) -1.05MB 0 bytes -100.0% 🗑️
9523.js (Deleted) -1.04MB 0 bytes -100.0% 🗑️
8049.js (Deleted) -548.34kB 0 bytes -100.0% 🗑️
1561.js (Deleted) -443.03kB 0 bytes -100.0% 🗑️
9269.js (Deleted) -366.77kB 0 bytes -100.0% 🗑️
5017.js (Deleted) -275.46kB 0 bytes -100.0% 🗑️
8413.js (Deleted) -218.92kB 0 bytes -100.0% 🗑️
8454.js (Deleted) -150.03kB 0 bytes -100.0% 🗑️
5151.js (Deleted) -121.83kB 0 bytes -100.0% 🗑️
6548.js (Deleted) -99.76kB 0 bytes -100.0% 🗑️
4236.js (Deleted) -98.52kB 0 bytes -100.0% 🗑️
5947.js (Deleted) -84.15kB 0 bytes -100.0% 🗑️
8478.js (Deleted) -77.8kB 0 bytes -100.0% 🗑️
5905.js (Deleted) -71.97kB 0 bytes -100.0% 🗑️
8548.js (Deleted) -67.9kB 0 bytes -100.0% 🗑️
6795.js (Deleted) -61.11kB 0 bytes -100.0% 🗑️
257.js (Deleted) -46.58kB 0 bytes -100.0% 🗑️
2300.js (Deleted) -43.71kB 0 bytes -100.0% 🗑️
5629.js (Deleted) -39.37kB 0 bytes -100.0% 🗑️
5103.js (Deleted) -37.06kB 0 bytes -100.0% 🗑️
7238.js (Deleted) -36.97kB 0 bytes -100.0% 🗑️
7390.js (Deleted) -34.31kB 0 bytes -100.0% 🗑️
8790.js (Deleted) -31.38kB 0 bytes -100.0% 🗑️
7153.js (Deleted) -30.44kB 0 bytes -100.0% 🗑️
7484.js (Deleted) -26.21kB 0 bytes -100.0% 🗑️
1058.js (Deleted) -26.05kB 0 bytes -100.0% 🗑️
2351.js (Deleted) -25.12kB 0 bytes -100.0% 🗑️
1010.js (Deleted) -24.59kB 0 bytes -100.0% 🗑️
6768.js (Deleted) -24.2kB 0 bytes -100.0% 🗑️
291.js (Deleted) -23.99kB 0 bytes -100.0% 🗑️
9567.js (Deleted) -22.21kB 0 bytes -100.0% 🗑️
8938.js (Deleted) -22.16kB 0 bytes -100.0% 🗑️
7244.js (Deleted) -20.92kB 0 bytes -100.0% 🗑️
4819.js (Deleted) -12.26kB 0 bytes -100.0% 🗑️
4781.js (Deleted) -11.95kB 0 bytes -100.0% 🗑️
776.js (Deleted) -11.55kB 0 bytes -100.0% 🗑️
8065.js (Deleted) -6.89kB 0 bytes -100.0% 🗑️
7000.js (Deleted) -5.23kB 0 bytes -100.0% 🗑️
9153.js (Deleted) -4.85kB 0 bytes -100.0% 🗑️
7067.js (Deleted) -1.47kB 0 bytes -100.0% 🗑️
1672.js (Deleted) -1.36kB 0 bytes -100.0% 🗑️
9597.js (Deleted) -1.36kB 0 bytes -100.0% 🗑️
9556.js (Deleted) -1.36kB 0 bytes -100.0% 🗑️
2180.js (Deleted) -1.27kB 0 bytes -100.0% 🗑️
3011.js (Deleted) -1.27kB 0 bytes -100.0% 🗑️
8247.js (Deleted) -1.27kB 0 bytes -100.0% 🗑️
2051.js (Deleted) -1.26kB 0 bytes -100.0% 🗑️
9980.js (Deleted) -1.26kB 0 bytes -100.0% 🗑️
4749.js (Deleted) -1.19kB 0 bytes -100.0% 🗑️
6998.js (Deleted) -1.18kB 0 bytes -100.0% 🗑️
4160.js (Deleted) -1.1kB 0 bytes -100.0% 🗑️
6468.js (Deleted) -1.1kB 0 bytes -100.0% 🗑️
9129.js (Deleted) -1.1kB 0 bytes -100.0% 🗑️
9445.js (Deleted) -971 bytes 0 bytes -100.0% 🗑️
2202.js (Deleted) -971 bytes 0 bytes -100.0% 🗑️
5097.js (Deleted) -971 bytes 0 bytes -100.0% 🗑️
4865.js (Deleted) -911 bytes 0 bytes -100.0% 🗑️
3331.js (Deleted) -905 bytes 0 bytes -100.0% 🗑️
7502.js (Deleted) -905 bytes 0 bytes -100.0% 🗑️
7560.js (Deleted) -904 bytes 0 bytes -100.0% 🗑️
5225.js (Deleted) -903 bytes 0 bytes -100.0% 🗑️
1938.js (Deleted) -902 bytes 0 bytes -100.0% 🗑️
388.js (Deleted) -902 bytes 0 bytes -100.0% 🗑️

Files in ../app/api/source-map/route.js:

  • ./src/frontmatter.ts → Total Size: 2.6kB

Files in ../app/sitemap.xml/route.js:

  • ./src/frontmatter.ts → Total Size: 2.6kB

App Routes Affected:

App Route Size Change Total Size Change (%)
/ -2.74MB 71.88kB -97.44%
/platform-redirect -2.71MB 73.73kB -97.35%
/[[...path]] -2.71MB 661.36kB -80.35%
view changes for bundle: sentry-docs-client-array-push

Assets Changed:

Asset Name Size Change Total Size Change (%)
static/chunks/pages/_app-*.js -3 bytes 882.71kB -0.0%
static/chunks/8321-*.js -3 bytes 425.87kB -0.0%
server/app/api/source-*.js (New) 267.83kB 267.83kB 100.0% 🚀
server/middleware-*.js -6.46kB 1.0kB -86.59%
server/middleware-*.js 6.46kB 7.46kB 645.5% ⚠️
app-*.json 412 bytes 5.01kB 8.96% ⚠️
static/chunks/app/api/ip-*.js 5 bytes 894 bytes 0.56%
static/chunks/app/_not-*.js 5 bytes 894 bytes 0.56%
static/chunks/app/api/source-*.js (New) 894 bytes 894 bytes 100.0% 🚀
static/ROiiEb3naxS9thr8es1OU/_buildManifest.js (New) 725 bytes 725 bytes 100.0% 🚀
static/ROiiEb3naxS9thr8es1OU/_ssgManifest.js (New) 77 bytes 77 bytes 100.0% 🚀
static/b8gBlc6fEKKyfnwVM7TK4/_buildManifest.js (Deleted) -684 bytes 0 bytes -100.0% 🗑️
static/b8gBlc6fEKKyfnwVM7TK4/_ssgManifest.js (Deleted) -77 bytes 0 bytes -100.0% 🗑️

- Extract lightweight frontmatter module to avoid bundling mdx-bundler
- Add aggressive outputFileTracingExcludes for build dependencies
- Update source-map API route to use lightweight imports
@jaffrepaul jaffrepaul marked this pull request as ready for review November 10, 2025 21:15
@@ -1,7 +1,7 @@
import type {MetadataRoute} from 'next';

import {getDevDocsFrontMatter, getDocsFrontMatter} from 'sentry-docs/frontmatter';
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Sitemap excludes critical API documentation.

The sitemap changed its import from sentry-docs/mdx to sentry-docs/frontmatter, but the new frontmatter.ts module doesn't include API-generated pages (like API categories and endpoints) that the original mdx.ts implementation adds. This causes the sitemap to exclude all dynamically-generated API documentation pages that were previously included.

Fix in Cursor Fix in Web

)
);

return allFrontMatter;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Frontmatter Collection Incomplete, Pages Vanish

The new getDocsFrontMatterUncached implementation only collects regular documentation files but completely omits the processing of common files for platforms and guides that the original src/mdx.ts implementation includes. This means thousands of pages generated from common source files (the core feature this PR is about) won't be included in the frontmatter collection, breaking the deduplication functionality and causing missing pages in the sitemap and source-map.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants