Upload the Big Replay

What you'll learn

Parts are 5 MiB–5 GiB (the last can be smaller), numbered 1–10,000; an object can reach about 50 TB (48.8 TiB).
Choose the part size from the file size: 10,000 parts is the ceiling.
Parts of an unfinished upload are billed until you complete or abort it.

Section 01

Three steps

CreateMultipartUpload returns an UploadId. Each UploadPart sends one numbered part and returns its ETag. CompleteMultipartUpload lists every part number with its ETag, and S3 stitches them into one object.

Parts can upload in any order and in parallel; a failed part is retried alone. AWS recommends multipart once objects reach about 100 MB.

Limit	Value
Part size	5 MiB to 5 GiB (last part: any size)
Parts per upload	1 to 10,000
Single PUT	Up to 5 GiB
Largest object	48.8 TiB (about 50 TB)

Limits from the Amazon S3 User Guide. The multipart ETag ends in -<number of parts>.

Section 02

Let lib-storage do it

Upload from @aws-sdk/lib-storage runs all three steps, uploads queueSize parts at a time (default 4) and reports progress. Its default part size is 5 MiB — fine for a 1 GiB file, too small for 80 GiB: 5 MiB × 10,000 is only about 49 GiB.

Work it out: 80 GiB ÷ 10,000 ≈ 8.2 MiB, so any part size from 9 MiB up works. Bigger parts mean fewer requests; smaller ones mean less to resend when a part fails.

Section 03

Don't pay for ghosts

An upload that's never completed or aborted leaves its parts behind. They don't appear in ListObjectsV2, but you pay for their storage. On failure, lib-storage aborts the upload by default (leavePartsOnError: false); code that uses the low-level commands must call AbortMultipartUpload itself.

As a safety net, add a lifecycle rule that aborts incomplete uploads after a few days — you'll write it in the next quest.

Practice

Your turn

The Sinnoh League final is an 80 GiB replay. Upload from @aws-sdk/lib-storage splits it into parts and sends several at once, but the part size here breaks S3's limits. Pick a part size that works, and make sure a failed upload doesn't leave billed parts behind.

Choose a partSize that keeps the upload within 10,000 parts, and stop keeping parts when an upload fails.

// @ts-nocheck
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import { createReadStream } from "node:fs";
const client = new S3Client({ region: "ap-south-1" });
const replaySizeBytes = 80 * 1024 ** 3; // 80 GiB
async function uploadReplay(path) {
  const upload = new Upload({
    client,
    params: {
      Bucket: "hoenn-pokedex-media",
      Key: "replays/2026/sinnoh-league-final.mp4",
      Body: createReadStream(path),
      ContentType: "video/mp4",
    },
    partSize: 5 * 1024 * 1024, // ⬅️ 5 MiB parts
    queueSize: 4,
    leavePartsOnError: true, // "keep the parts so we can debug failures"
  });
  upload.on("httpUploadProgress", (p) => console.log(p.loaded, "/", p.total));
  await upload.done();
}

// @ts-nocheck
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import { createReadStream } from "node:fs";

const client = new S3Client({ region: "ap-south-1" });

const replaySizeBytes = 80 * 1024 ** 3; // 80 GiB

async function uploadReplay(path) {
  const upload = new Upload({
    client,
    params: {
      Bucket: "hoenn-pokedex-media",
      Key: "replays/2026/sinnoh-league-final.mp4",
      Body: createReadStream(path),
      ContentType: "video/mp4",
    },
    partSize: 5 * 1024 * 1024, // ⬅️ 5 MiB parts
    queueSize: 4,
    leavePartsOnError: true, // "keep the parts so we can debug failures"
  });

  upload.on("httpUploadProgress", (p) => console.log(p.loaded, "/", p.total));
  await upload.done();
}

Ctrl/⌘ Enter to run · simulated, no AWS callsSimulated — no AWS calls

Check

Check your understanding

Quick Quiz · one question, as many tries as you need.

A 200 GiB file must be uploaded. What is the smallest whole-MiB part size that works?

Choose one answer.