If you only read this
- Never push file bytes through your API. Presigned part URLs send them straight to S3 or equivalent.
- Checkpoint every part in SQLite. A kill costs at most one part.
- iOS needs a background URLSession with file-based tasks; Android needs WorkManager or a user-initiated data transfer job.
01Shape of the problem
A 4 GB video on a moving train sees minutes of signal, then none, then a different network. A single HTTP request cannot survive that. The design has to assume the upload will be interrupted dozens of times and cost almost nothing each time.
Four ideas combine to make it work: break the file into independent parts, upload each part with its own URL, record which parts are done, and verify the whole thing at the end.
02Protocol, end to end
Notice who talks to whom. Your API never sees the bytes. It creates the upload, hands out time-limited URLs and, at the end, completes and verifies it. That is what lets 5 million phones upload at once without your servers becoming the bottleneck.
03Hashing, parts and limits
Hash the file while reading it in chunks, so memory stays flat. Send the SHA-256 at the start. The server can answer 'already have it', which turns a duplicate upload into a no-op, and uses the hash at the end to verify integrity.
Choose the part size from the file size and the network. S3 multipart parts must be at least 5 MiB (except the last) and an upload can have at most 10,000 parts, so 8 to 16 MiB works well for gigabyte files: 4 GB at 8 MiB is 512 parts. Smaller parts lose less when interrupted and larger parts have less overhead, so adapt: start at 8 MiB and move up on Wi-Fi.
Upload state the transfer manager persists
1CREATE TABLE upload (2 id TEXT PRIMARY KEY, path TEXT NOT NULL, size INTEGER NOT NULL, sha256 TEXT NOT NULL,3 upload_id TEXT, part_size INTEGER NOT NULL, state TEXT NOT NULL, -- queued|hashing|uploading|completing|done|failed4 attempts INTEGER NOT NULL DEFAULT 0, next_try_at INTEGER5);6CREATE TABLE upload_part (7 upload_id TEXT NOT NULL, n INTEGER NOT NULL, etag TEXT, bytes INTEGER NOT NULL,8 PRIMARY KEY (upload_id, n) -- a row with an etag is a finished part9);04Android: who is allowed to run this
- WorkManager runs resumable work with constraints (network, battery) and survives reboots. Use a foreground service with a progress notification for work the user is watching.
- Android 14 added user-initiated data transfer jobs for large transfers the user started. They are exempt from the usual background limits, require a notification, and fit a 4 GB upload.
- Doze and app standby delay jobs but do not cancel them. Some manufacturers add their own task killers, so treat background execution as best effort and resume instantly when the app opens.
- Use OkHttp with a streaming request body for each part, reading from the file with an offset. Do not load a part into memory.
05iOS: the background URLSession rules
- Large background uploads must be file-based: URLSession.uploadTask(with:fromFile:) in a background configuration. The system process performs the transfer and keeps going when your app is suspended or terminated.
- Create one task per part. Write each part to its own temporary file first (or use a file with byte ranges through a custom stream where supported), because tasks upload a file, not a slice of memory.
- When a task finishes, iOS relaunches your app in the background and calls the application delegate's handler for the session identifier. Reconnect the session with the same identifier, handle the delegate callbacks, then call the completion handler.
- Set isDiscretionary for non-urgent uploads so the system picks good moments, and decide explicitly about allowsCellularAccess and allowsExpensiveNetworkAccess.
- BGTaskScheduler is for deciding what to do next (finalising, retrying, hashing). It is not the transfer engine.
06Downloads, Range requests and tus
Resumable downloads use HTTP Range requests. Send If-Range with the ETag you saw, so that if the file changed on the server you restart instead of splicing two versions together. Write to a .part file, verify the hash, then rename atomically.
If you do not want to run multipart logic yourself, the open tus protocol and managed services such as Cloudflare Stream or Mux handle resumable uploads with a standard client. Build it yourself when you need control over cost, residency or integrity checks; adopt tus or a vendor when upload is not your differentiator.
Resumable download sketch
1val have = partFile.length()2val req = Request.Builder().url(url)3 .header("Range", "bytes=" + have + "-")4 .header("If-Range", savedEtag) // restart if the object changed5 .build()6client.newCall(req).execute().use { resp ->7 when (resp.code) {8 206 -> append(resp.body!!.source(), partFile) // continue9 200 -> { partFile.delete(); append(resp.body!!.source(), partFile) } // changed: start over10 416 -> verifyAndRename(partFile) // already complete11 else -> throw RetryableHttp(resp.code)12 }13}07Tested code, explained
Cutting a file into parts
This is simple arithmetic, and getting the last part wrong corrupts uploads.
1data class Part(val no: Int, val offset: Long, val length: Long)2 3fun planParts(size: Long, partSize: Long = 8 * MIB): List<Part> {4 val count = ((size + partSize - 1) / partSize).toInt()5 return List(count) { i ->6 val offset = i * partSize7 Part(i + 1, offset, minOf(partSize, size - offset))8 }9}(size + partSize - 1) / partSize- Rounds up, so 412 MB makes 52 parts of 8 MB.
minOf(partSize, size - offset)- The last part is usually shorter.
Part(i + 1, ...)- Object stores number parts from 1.
Compiled and tested by npm run test:mobile.
Sending parts that survive being killed
Arjun's upload. The receipts map stands for a table on disk, so a killed app loses almost nothing.
1/**2 * Sends only what the table says is missing. [receipts] stands for the transfer_part3 * table: a part number maps to its ETag once that part succeeds.4 */5class PartUploader(6 private val parts: List<Part>,7 private val receipts: MutableMap<Int, String>,8 private val freshUrls: suspend (List<Part>) -> Map<Int, String>,9 private val send: suspend (Part, url: String) -> String,10 private val parallel: Int = 4,11) {12 val remaining: List<Part> get() = parts.filter { it.no !in receipts }13 14 suspend fun run() {15 var urls = freshUrls(remaining)16 while (remaining.isNotEmpty()) {17 val outcomes = coroutineScope {18 remaining.take(parallel).map { part ->19 async { part to runCatching { send(part, urls.getValue(part.no)) } }20 }.awaitAll()21 }22 // Save every success first, so a failure next to it costs nothing.23 outcomes.forEach { (part, r) -> r.onSuccess { etag -> receipts[part.no] = etag } }24 val failure = outcomes.firstNotNullOfOrNull { (_, r) -> r.exceptionOrNull() }25 when (failure) {26 null -> Unit27 is UrlExpired -> urls = freshUrls(remaining) // only the parts still missing28 else -> throw failure29 }30 }31 }32 33 /** The body of POST /files/complete, in part order. */34 fun manifest(): List<Pair<Int, String>> = receipts.toSortedMap().map { (no, etag) -> no to etag }35}parts.filter { it.no !in receipts }- What is left. This one line is why resuming needs no special case: a fresh start and a resume run the same code.
runCatching { send(...) }- One failed part doesn't cancel the others in flight.
outcomes.forEach { ... receipts[...] = etag }- Saves every success before looking at failures, so finished work always counts.
is UrlExpired -> urls = freshUrls(remaining)- Fetches new URLs only for the parts still missing, then carries on.
else -> throw failure- Anything else ends this run. The next run picks up from the receipts.
Compiled and tested by npm run test:mobile.
08Interview questions
Why multipart with presigned URLs instead of one POST to your API?
The app is killed at part 301 of 512. What happens?
How do you stop a user uploading the same video twice?
Why can't iOS upload a part straight from memory in the background?
How do you handle a flaky captive-portal Wi-Fi that returns 200 with HTML?
Practise it: