So the other day, I had the pleasure of running into this lovely error when deploying my Next.js website to Vercel:
The nice thing about this error is that you only get it when you deploy your website, and only after the issue has been compounding for a while.
The error message is pretty clear, as it tells you which serverless function exceeds the limit, and what the largest dependencies are, but in my case, it was a bit of a mystery since all of my functions were exceeding the limit, and the largest dependencies were all the same.
Here's the build output for reference:
[11:18:14.653] Running build in Washington, D.C., USA (East) – iad1
[11:18:14.874] Cloning github.com/ykadosh/yoavik.com (Branch: master, Commit: 99c264e)
[11:18:14.982] Previous build caches not available
[11:18:29.516] Cloning completed: 14.641s
[11:18:29.925] Running "vercel build"
[11:18:30.352] Vercel CLI 41.2.2
[11:18:30.756] Installing dependencies...
[11:18:38.927] npm warn deprecated babel-eslint@10.1.0: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
[11:18:51.662]
[11:18:51.662] added 852 packages in 21s
[11:18:51.663]
[11:18:51.663] 314 packages are looking for funding
[11:18:51.663] run `npm fund` for details
[11:18:51.688] Detected Next.js version: 14.2.15
[11:18:51.700] Running "npm run build"
[11:18:51.899]
[11:18:51.899] > yoavik.com@1.0.0 build
[11:18:51.899] > next build
[11:18:51.900]
[11:18:52.860] Attention: Next.js now collects completely anonymous telemetry regarding usage.
[11:18:52.861] This information is used to shape Next.js' roadmap and prioritize features.
[11:18:52.861] You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
[11:18:52.861] https://nextjs.org/telemetry
[11:18:52.861]
[11:18:52.919] ▲ Next.js 14.2.15
[11:18:52.920]
[11:18:52.920] Linting and checking validity of types ...
[11:18:53.167] Creating an optimized production build ...
[11:19:36.689] ✓ Compiled successfully
[11:19:36.689] Collecting page data ...
[11:19:38.558] Generating static pages (0/40) ...
[11:19:39.262] Generating static pages (10/40)
[11:19:40.096] Generating static pages (20/40)
[11:19:40.926] Generating static pages (30/40)
[11:19:42.285] ✓ Generating static pages (40/40)
[11:19:49.504] Finalizing page optimization ...
[11:19:49.505] Collecting build traces ...
[11:19:49.664] ⚠ `config.analyticsId` is deprecated and will be removed in next major version. Read more: https://nextjs.org/docs/messages/deprecated-analyticsid
[11:19:53.611]
[11:19:53.616] Route (pages) Size First Load JS
[11:19:53.617] ┌ ● / 10.5 kB 150 kB
[11:19:53.617] ├ └ css/77bab511c8c3e21a.css 1.29 kB
[11:19:53.617] ├ /_app 0 B 97.1 kB
[11:19:53.617] ├ ○ /404 3.84 kB 109 kB
[11:19:53.618] ├ ƒ /api/likes/[id] 0 B 97.1 kB
[11:19:53.618] ├ ƒ /api/likes/increment 0 B 97.1 kB
[11:19:53.618] ├ ƒ /api/newsletter/add 0 B 97.1 kB
[11:19:53.618] ├ ƒ /api/twitter/popular 0 B 97.1 kB
[11:19:53.618] ├ ƒ /api/views/[id] 0 B 97.1 kB
[11:19:53.619] ├ ● /experiments 1.46 kB 115 kB
[11:19:53.619] ├ ● /fabrication 1.2 kB 119 kB
[11:19:53.623] ├ ○ /fabrication/band-saw-cabinet 2.05 kB 185 kB
[11:19:53.623] ├ ○ /fabrication/book-shelves 4.05 kB 187 kB
[11:19:53.623] ├ ○ /fabrication/drill-press-cabinet 4.76 kB 188 kB
[11:19:53.623] ├ ○ /fabrication/friendship-bench 5.9 kB 189 kB
[11:19:53.623] ├ └ css/1beea7a62195c237.css 2.68 kB
[11:19:53.623] ├ ○ /fabrication/lego-cabinet 6.81 kB 190 kB
[11:19:53.623] ├ ○ /fabrication/office-desk 4.7 kB 188 kB
[11:19:53.624] ├ ○ /fabrication/planter-box 5.46 kB 189 kB
[11:19:53.624] ├ ○ /fabrication/rabbit-hutch 4.01 kB 187 kB
[11:19:53.624] ├ ○ /fabrication/tic-tac-toe 2.91 kB 186 kB
[11:19:53.624] ├ ○ /fabrication/waterfall-tables 8.33 kB 191 kB
[11:19:53.624] ├ ○ /fabrication/wine-glass-holder 1.89 kB 185 kB
[11:19:53.624] ├ ○ /fabrication/wooden-chandelier 3.71 kB 187 kB
[11:19:53.624] ├ ○ /fabrication/wooden-playhouse 10.4 kB 193 kB
[11:19:53.624] ├ ○ /fabrication/workbench 4.34 kB 187 kB
[11:19:53.625] ├ ○ /fabrication/yard-fence 4.52 kB 188 kB
[11:19:53.625] ├ ● /posts 1.39 kB 115 kB
[11:19:53.625] ├ └ css/d3ec71a54ac08158.css 679 B
[11:19:53.625] ├ ○ /posts/canvas-path-animation 1.63 kB 176 kB
[11:19:53.625] ├ ○ /posts/embedding-tweets 2.69 kB 178 kB
[11:19:53.625] ├ ○ /posts/generate-toc-mdx 5.4 kB 180 kB
[11:19:53.625] ├ ○ /posts/how-i-built-my-blog 648 B 175 kB
[11:19:53.626] ├ ○ /posts/playground 4.43 kB 379 kB
[11:19:53.626] ├ └ css/d1c9d54da62da662.css 2.28 kB
[11:19:53.626] ├ ○ /posts/randomizing-svg-paths 821 B 176 kB
[11:19:53.626] ├ ƒ /rss.xml 244 B 97.4 kB
[11:19:53.626] ├ ƒ /search 3.71 kB 122 kB
[11:19:53.626] ├ └ css/f7900d16be8a4b87.css 679 B
[11:19:53.626] ├ ƒ /sitemap.txt 248 B 97.4 kB
[11:19:53.627] ├ ● /snippets 2.17 kB 116 kB
[11:19:53.627] ├ ○ /snippets/3d-text 2.61 kB 177 kB
[11:19:53.627] ├ └ css/de6589cdc3b937a2.css 3.24 kB
[11:19:53.627] ├ ○ /snippets/client-only 1.18 kB 176 kB
[11:19:53.627] ├ ○ /snippets/codepen 2.36 kB 177 kB
[11:19:53.627] ├ ○ /snippets/image-with-fallback 6.82 kB 381 kB
[11:19:53.628] ├ ○ /snippets/media-query-mixin 1.98 kB 177 kB
[11:19:53.628] ├ ○ /snippets/mouse-tracker 5.38 kB 380 kB
[11:19:53.628] ├ ○ /snippets/use-animation-frame 1.08 kB 176 kB
[11:19:53.628] ├ ○ /snippets/use-document-event 2.28 kB 177 kB
[11:19:53.628] ├ ○ /snippets/use-global-state 6.63 kB 381 kB
[11:19:53.628] ├ ○ /snippets/use-intersection-observer 3.07 kB 178 kB
[11:19:53.628] ├ ○ /snippets/use-perishable 6.95 kB 381 kB
[11:19:53.629] └ ○ /snippets/use-value 4.05 kB 378 kB
[11:19:53.629] + First Load JS shared by all 105 kB
[11:19:53.629] ├ chunks/framework-945b357d4a851f4b.js 44.9 kB
[11:19:53.629] ├ chunks/main-5ae912dde92706cf.js 32.6 kB
[11:19:53.629] ├ chunks/pages/_app-31da04a866cf9ef8.js 18 kB
[11:19:53.629] └ other shared chunks (total) 9.13 kB
[11:19:53.630]
[11:19:53.634] ○ (Static) prerendered as static content
[11:19:53.634] ● (SSG) prerendered as static HTML (uses getStaticProps)
[11:19:53.634] ƒ (Dynamic) server-rendered on demand
[11:19:53.634]
[11:19:53.789] Traced Next.js server files in: 33.243ms
[11:19:56.275] Warning: Max serverless function size of 250 MB uncompressed reached
[11:19:56.276] Serverless Function's page: experiments.js
[11:19:56.281] Large Dependencies Uncompressed size
[11:19:56.282] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.282] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.282] pages/fabrication/planter-box 24.06 MB
[11:19:56.282] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.282] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.282] pages/fabrication/yard-fence 13.71 MB
[11:19:56.282] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.282] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.283] pages/fabrication/workbench 11.41 MB
[11:19:56.283] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.283] pages/fabrication/office-desk 10.37 MB
[11:19:56.283] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.283] pages/fabrication/book-shelves 7.75 MB
[11:19:56.283] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.283] node_modules/react-dom/cjs 1.64 MB
[11:19:56.283] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.284] node_modules/next/dist 961.97 KB
[11:19:56.284]
[11:19:56.284] All dependencies 254.39 MB
[11:19:56.284] Serverless Function's page: fabrication.js
[11:19:56.289] Large Dependencies Uncompressed size
[11:19:56.289] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.289] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.289] pages/fabrication/planter-box 24.06 MB
[11:19:56.289] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.289] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.289] pages/fabrication/yard-fence 13.71 MB
[11:19:56.289] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.290] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.290] pages/fabrication/workbench 11.41 MB
[11:19:56.290] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.290] pages/fabrication/office-desk 10.37 MB
[11:19:56.290] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.290] pages/fabrication/book-shelves 7.75 MB
[11:19:56.290] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.290] node_modules/react-dom/cjs 1.64 MB
[11:19:56.290] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.291] node_modules/next/dist 961.97 KB
[11:19:56.291]
[11:19:56.291] All dependencies 254.41 MB
[11:19:56.291] Serverless Function's page: index.js
[11:19:56.294] Large Dependencies Uncompressed size
[11:19:56.294] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.294] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.295] pages/fabrication/planter-box 24.06 MB
[11:19:56.295] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.295] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.295] pages/fabrication/yard-fence 13.71 MB
[11:19:56.295] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.296] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.296] pages/fabrication/workbench 11.41 MB
[11:19:56.296] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.297] pages/fabrication/office-desk 10.37 MB
[11:19:56.297] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.297] pages/fabrication/book-shelves 7.75 MB
[11:19:56.297] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.297] node_modules/react-dom/cjs 1.64 MB
[11:19:56.297] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.297] node_modules/next/dist 961.97 KB
[11:19:56.298]
[11:19:56.298] All dependencies 254.84 MB
[11:19:56.300] Serverless Function's page: posts.js
[11:19:56.300] Large Dependencies Uncompressed size
[11:19:56.301] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.301] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.301] pages/fabrication/planter-box 24.06 MB
[11:19:56.301] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.301] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.301] pages/fabrication/yard-fence 13.71 MB
[11:19:56.301] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.302] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.302] pages/fabrication/workbench 11.41 MB
[11:19:56.302] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.302] pages/fabrication/office-desk 10.37 MB
[11:19:56.302] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.302] pages/fabrication/book-shelves 7.75 MB
[11:19:56.302] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.302] node_modules/react-dom/cjs 1.64 MB
[11:19:56.303] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.303] node_modules/next/dist 961.97 KB
[11:19:56.303]
[11:19:56.303] All dependencies 254.39 MB
[11:19:56.305] Serverless Function's page: rss.xml.js
[11:19:56.307] Large Dependencies Uncompressed size
[11:19:56.307] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.307] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.307] pages/fabrication/planter-box 24.06 MB
[11:19:56.307] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.307] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.307] pages/fabrication/yard-fence 13.71 MB
[11:19:56.308] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.308] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.308] pages/fabrication/workbench 11.41 MB
[11:19:56.308] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.308] pages/fabrication/office-desk 10.37 MB
[11:19:56.308] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.308] pages/fabrication/book-shelves 7.75 MB
[11:19:56.308] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.309] node_modules/react-dom/cjs 1.64 MB
[11:19:56.309] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.309] node_modules/next/dist 961.97 KB
[11:19:56.309]
[11:19:56.309] All dependencies 254.43 MB
[11:19:56.314] Serverless Function's page: search.js
[11:19:56.329] Large Dependencies Uncompressed size
[11:19:56.330] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.330] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.330] pages/fabrication/planter-box 24.06 MB
[11:19:56.330] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.330] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.331] pages/fabrication/yard-fence 13.71 MB
[11:19:56.331] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.331] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.331] pages/fabrication/workbench 11.41 MB
[11:19:56.331] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.331] pages/fabrication/office-desk 10.37 MB
[11:19:56.331] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.331] pages/fabrication/book-shelves 7.75 MB
[11:19:56.332] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.332] node_modules/react-dom/cjs 1.64 MB
[11:19:56.332] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.332] node_modules/next/dist 961.97 KB
[11:19:56.332]
[11:19:56.332] All dependencies 254.41 MB
[11:19:56.333] Serverless Function's page: sitemap.txt.js
[11:19:56.337] Large Dependencies Uncompressed size
[11:19:56.337] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.338] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.338] pages/fabrication/planter-box 24.06 MB
[11:19:56.338] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.338] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.338] pages/fabrication/yard-fence 13.71 MB
[11:19:56.338] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.338] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.338] pages/fabrication/workbench 11.41 MB
[11:19:56.339] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.339] pages/fabrication/office-desk 10.37 MB
[11:19:56.339] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.339] pages/fabrication/book-shelves 7.75 MB
[11:19:56.339] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.339] node_modules/react-dom/cjs 1.64 MB
[11:19:56.339] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.339] node_modules/next/dist 961.97 KB
[11:19:56.340]
[11:19:56.340] All dependencies 254.34 MB
[11:19:56.342] Serverless Function's page: snippets.js
[11:19:56.346] Large Dependencies Uncompressed size
[11:19:56.346] pages/fabrication/wooden-playhouse 63.08 MB
[11:19:56.346] pages/fabrication/waterfall-tables 25.86 MB
[11:19:56.346] pages/fabrication/planter-box 24.06 MB
[11:19:56.346] pages/fabrication/lego-cabinet 23.41 MB
[11:19:56.346] pages/fabrication/drill-press-cabinet 21.03 MB
[11:19:56.346] pages/fabrication/yard-fence 13.71 MB
[11:19:56.346] pages/fabrication/friendship-bench 13.12 MB
[11:19:56.346] pages/fabrication/wooden-chandelier 12.3 MB
[11:19:56.346] pages/fabrication/workbench 11.41 MB
[11:19:56.346] pages/fabrication/rabbit-hutch 10.55 MB
[11:19:56.346] pages/fabrication/office-desk 10.37 MB
[11:19:56.346] pages/fabrication/tic-tac-toe 8.67 MB
[11:19:56.346] pages/fabrication/book-shelves 7.75 MB
[11:19:56.346] pages/fabrication/band-saw-cabinet 2.27 MB
[11:19:56.346] node_modules/react-dom/cjs 1.64 MB
[11:19:56.346] pages/fabrication/wine-glass-holder 1.12 MB
[11:19:56.346] node_modules/next/dist 961.97 KB
[11:19:56.346]
[11:19:56.347] All dependencies 254.39 MB
[11:19:56.347] Max serverless function size was exceeded for 8 functions
[11:19:56.359] Created all serverless functions in: 2.570s
[11:19:56.396] Collected static files (public/, static/, .next/static): 31.907ms
[11:19:56.530] Build Completed in /vercel/output [1m]
[11:19:56.920] Deploying outputs...
[11:21:29.071] Error: A Serverless Function has exceeded the unzipped maximum size of 250 MB. : https://vercel.link/serverless-function-size
[11:21:32.679]As you can see, the largest dependencies are all coming from the fabrication directory, because there are a lot of images in there, and the serverless function is trying to bundle them all. But why?
#Debugging locally
So the first step in debugging this issue is to try and reproduce it locally.
According to the build output, the command that failed was vercel build, so I tried running it
locally to see if I could reproduce the issue.
This was the first time I was running this command, so I had to install the Vercel CLI first:
npm i -g vercelAfter installing the CLI, I ran the build command, and got the same output as the one above.
I focused on the snippets.js page, since it was the last one.
I opened the file and started by removing all of the imports, and then adding them back one by one
to see which one was causing the issue. Each time I ran the vercel build command to see if snippets.js
was still appearing in the output.
Once I found the problematic import, I opened that file and repeated the same process. It was a bit of a rabbit hole, but eventually I found the issue. It was this function:
export const getPosts = async () => {
const metas = require.context('../pages/posts', true, /\.\/[^/]+\/index\.meta\.js$/);
const posts = [];
for (let path of metas.keys()) {
const { meta } = await metas(path);
if (meta.published || process.env.NODE_ENV === 'development') {
meta.url = `/posts/${path.replace('/index.meta.js', '')}`;
const PAGES_DIR = join(process.cwd(), 'pages');
const content = await fs.readFile(join(PAGES_DIR, 'posts', path.replace('meta.js', 'page.mdx')), 'utf8');
meta.readingTime = await getReadingTime(content);
posts.push(meta);
}
}
return posts;
}The purpose of this function is to read the metadata of all the posts in the pages/posts directory, and to enrich it
with additional information like reading time. The reading time is calculated based on the size of the content,
which we get by reading the file. To read the file, I needed the absolute path, which I was calculating
by joining the current working directory (process.cwd()) with the relative path of the file.
This is a dynamic import with a variable path, and that was the root cause of the issue.
#The problem
Dynamic imports with variable paths are technically impossible, because the bundling is done at build time, while the path is only known at runtime. This means that Webpack doesn't know which files to include in the bundle, and it ends up including all the files in the directory.
In my case, this was a fully dynamic path, so it included all
the directories, including the fabrication directory, which contains a lot of images.
That's why the serverless function was exceeding the limit.
From Webpack's documentation:
It is not possible to use a fully dynamic import statement, such as
import(foo). Becausefoocould potentially be any path to any file in your system or project. Theimport()must contain at least some information about where the module is located. Bundling can be limited to a specific directory or set of files so that when you are using a dynamic expression - every module that could potentially be requested on animport()call is included.
https://webpack.js.org/api/module-methods/#dynamic-expressions-in-import
The solution is to limit the scope of the dynamic import to a specific directory, by providing the beginning of the path. So by changing a single line, the issue was fixed:
const content = await fs.readFile(join('pages/posts', path.replace('meta.js', 'page.mdx')), 'utf8');Now, only the files under the pages/posts directory are included in the bundle, and the serverless function
is no longer exceeding the limit. But it was only a matter of time before I ran into the same issue again,
because I was still using a dynamic import to read the metadata files.
#Over-engineering
This brings me to a broader conclusion about over-engineering.
The reason I had to use require.context and fs.readFile is because I wanted to avoid having to import and enrich all the files manually.
I wanted to be able to add a new post, and have the rest of the code automatically detect it.
But in reality, this is not a problem at all. I can just import the file manually when I add a new post, and if I forget to do it,
I'll see that it's missing in my website.
This is a much simpler solution, and it doesn't require me to use any hacks like require.context.
While the original solution was supposed to save me time, it ended up costing me a lot of time in debugging.
If you still want to automatically include all files in a directory, or produce some output based on file structure, you can run a script at build time that will generate an index file with all the imports, or even have it read the content and generate the metadata automatically.