Building a Photo Vault with Multipart Upload
Learn how to create a complete photo vault interface that displays existing user images and enables reliable large file uploads using advanced multipart upload techniques.
This comprehensive tutorial teaches both the concepts and implementation details, providing a complete working example that demonstrates best practices for handling file uploads in production applications.
Understanding Multipart Upload
What is Multipart Upload?
Imagine you need to mail a 500-page book to someone, but the postal service only accepts packages up to 50 pages. You could:
- Regular approach: Try to force the entire book into one package (fails)
- Multipart approach: Split the book into 10 packages of 50 pages each, mail them separately, then have the recipient reassemble them
Multipart upload works the same way with files. Instead of sending one massive HTTP request with your entire file, you:
- Split the file into smaller chunks (typically 5-10MB each)
- Upload each chunk independently using signed URLs
- Combine the chunks server-side into the original file
Why Use Multipart Upload?
๐ Performance Benefits
- Chunks upload in parallel, utilizing full bandwidth
- Faster overall upload times for large files
- Better resource utilization
๐ Reliability Improvements
- If one chunk fails, only that chunk needs retry (not the entire file)
- Network interruptions donโt restart the entire upload
- Automatic retry logic for transient failures
๐ Better User Experience
- Granular progress tracking (chunk-by-chunk)
- Users can see exactly how much has uploaded
- Cancel/resume functionality
๐๏ธ Technical Advantages
- No single request size limits (overcome HTTP timeouts)
- Reduced server memory usage (chunks processed individually)
- Improved error isolation and debugging
When Should You Use Multipart Upload?
โ Use multipart upload when:
- Files are larger than 10MB
- Users are on unreliable connections (mobile networks)
- You need detailed progress feedback
- Building production applications with user-generated content
- Handling media files (images, videos, audio)
โ Regular upload is fine when:
- Files are small (< 5MB)
- Network is reliable and fast
- Simple use cases without progress requirements
- Prototypes or internal tools
How Multipart Upload Works (The 3-Phase Flow)
Understanding this flow is crucial for implementing multipart upload correctly:
Phase 1: Create Upload Session (Setup)
- Tell the server you want to upload a file
- Server creates a media record in database
- Server starts a multipart upload session
- You receive a
mediaUuidanduploadIdfor tracking
Phase 2: Upload Parts (The Heavy Lifting)
- Split your file into chunks (weโll use 10MB chunks)
- For each chunk:
- Request a signed URL for that specific part number
- Upload the chunk directly to cloud storage using the signed URL
- Save the ETag returned by storage (proof of successful upload)
Phase 3: Complete Upload Session (Assembly)
- Tell the server all chunks are uploaded
- Provide the list of ETags to prove which chunks were uploaded
- Server combines chunks into the final file
- File enters processing pipeline (thumbnail generation, etc.)
What Youโll Build
By the end of this tutorial, youโll have a complete photo vault featuring:
๐ฑ Photo Grid Interface
- Responsive grid that adapts to screen size
- Displays existing vault images using optimized thumbnails
- Infinite scroll pagination for large image collections
- Loading states and graceful error handling
โฌ๏ธ Advanced Upload System
- Multipart upload with 10MB chunking
- Real-time progress tracking with visual feedback
- Automatic retry logic for failed chunks
- Upload queue management (multiple files simultaneously)
- File validation and error handling
๐ก๏ธ Production-Ready Features
- OAuth authentication integration
- Proper error boundaries and user feedback
- Mobile-responsive design
- Security best practices
- Performance optimizations
Learning Objectives
After completing this tutorial, youโll understand:
- How multipart upload works under the hood
- When and why to use multipart vs regular uploads
- How to implement chunked file upload with progress tracking
- OAuth authentication patterns for API access
- React patterns for handling async file operations
- Error handling strategies for network operations
- Performance optimization techniques for file uploads
Before You Start
Complete the Quickstart Guide first to set up OAuth authentication, environment variables, and your basic Fanvue app. This tutorial builds on that foundation and focuses specifically on multipart upload implementation.
Prerequisites
Since youโve completed the Quickstart Guide, you should have:
- โ A working Next.js app with Fanvue OAuth authentication
- โ
Environment variables configured (
OAUTH_CLIENT_ID,OAUTH_CLIENT_SECRET, etc.) - โ
Basic
read:selfscope working for user authentication
Additional Scopes Required
Add these scopes to your existing OAUTH_SCOPES in .env.local:
Make sure to update your app configuration in the Fanvue Developer area to include the
read:media and write:media scopes.
Technical Requirements
- File API Knowledge: Understanding of JavaScript File objects and Blob handling
- React Patterns: Experience with hooks, async state management, and component patterns
Ready to dive in? Letโs start by understanding how to fetch and display existing vault images, then build up to the complete multipart upload system.
Step 1: Fetch Existing Vault Images
Before we dive into multipart uploads, letโs start with something simpler: displaying existing images in the userโs vault. This will help us understand the API structure and build the foundation for our upload system.
Understanding the Media API
The Fanvue API organizes user content into media items with different variants (sizes/formats). For our vault, weโll focus on:
- Media Type:
image(filtering out videos and audio) - Variant Type:
thumbnail_gallery(optimized for grid display)
Building the API Client
Our API client handles OAuth authentication and provides methods for media operations. Notice how we use Bearer token authentication:
Whatโs happening here:
- OAuth Authentication: We use
Authorization: Bearer ${token} - Error Handling: Structured error parsing with fallbacks
- Flexibility: Support for filtering by type, status, variants, and pagination
- Type Safety: Full TypeScript interfaces for all API responses
Building the Photo Grid Component
Now letโs create a React component that displays user images in a responsive grid. This component demonstrates pagination, loading states, and error handling patterns youโll use throughout the vault.
Key Features Explained:
- OAuth Authentication: Uses
accessTokenprop for secure authentication - Responsive Grid: Adapts from 2 to 6 columns based on screen size
- Lazy Loading: Images load only when theyโre about to be visible
- State Management: Separate loading states for initial load vs. pagination
- Error Handling: Comprehensive error states with retry functionality
- Performance: Optimized with
useCallbackto prevent unnecessary re-renders
Now that you understand how to display existing images, letโs move on to the exciting part: implementing multipart upload!
Step 2: Implement Multipart Upload
Hereโs where we get to the core of this tutorial. Weโll build a complete multipart upload system that handles large files efficiently, provides real-time progress feedback, and gracefully handles errors.
Understanding the Upload Flow (Again, but with Code)
Before we dive into implementation, letโs revisit the 3-phase flow with more technical detail:
Phase 1: Initiate Upload
- Client tells server: โI want to upload a 50MB image called โvacation.jpgโโ
- Server creates a database record for the media
- Server tells cloud storage: โPrepare for a multipart uploadโ
- Server returns:
{mediaUuid: "abc-123", uploadId: "xyz-789"}
Phase 2: Upload Parts
- Client splits file into 10MB chunks (5 chunks for our 50MB file)
- For each chunk:
- Client asks: โWhere do I upload chunk 3?โ
- Server returns a signed URL (valid for ~15 minutes)
- Client uploads chunk directly to cloud storage
- Storage returns an ETag (proof of successful upload)
Phase 3: Finalize Upload
- Client tells server: โAll chunks uploaded, here are the ETagsโ
- Server tells storage: โCombine all chunks into final fileโ
- Storage combines chunks and confirms success
- Server starts processing pipeline (thumbnail generation, etc.)
Enhanced API Client with Upload Methods
Letโs extend our API client to support the multipart upload flow:
Whatโs happening in these upload methods:
- createUploadSession: Creates media record and starts multipart session
- getPartUrl: Gets signed URL for uploading a specific chunk using RESTful URL structure
- uploadPart: Uploads chunk directly to cloud storage (bypasses our API)
- completeUploadSession: Tells server to combine all chunks into final file using PATCH method
Error Handling for Upload Operations
Network operations can fail in many ways, especially with large file uploads. Letโs add robust error handling:
Building the Upload Progress Component
Before we build the main uploader, letโs create a component that shows upload progress. This provides immediate user feedback and makes the upload feel responsive:
This progress component handles all the visual feedback during upload. The key features are:
- Status-based styling: Different colors and icons for each state
- Progress calculation: Smooth progress bar with minimum 2% width for visibility
- Interactive elements: Cancel and retry buttons when appropriate
- Educational messaging: Explains what โprocessingโ means to users
Building the Main Upload Component
Now comes the centerpiece: a React component that handles the complete multipart upload flow. This component coordinates file chunking, parallel uploads, progress tracking, and error recovery.
Key Implementation Details:
- Chunking Strategy: Files are split into 10MB chunks for optimal balance of reliability and performance
- Concurrency Control: Maximum 3 concurrent uploads to avoid overwhelming the server
- Progress Tracking: Real-time progress updates as chunks complete
- Error Recovery: Automatic retry for network issues and rate limits
- User Control: Users can cancel uploads and retry failed ones
- Queue Management: Multiple files are queued and processed efficiently
Understanding the Upload Logic
Letโs break down the complex upload logic step by step:
File Validation
- Checks file type matches accepted formats
- Ensures file size is within limits
- Provides clear error messages for invalid files
Upload Orchestration
- Initiate: Create media record and get upload session ID
- Chunk Processing: Split file and upload chunks in parallel batches
- Progress Tracking: Update UI as each chunk completes
- Finalization: Combine chunks server-side and start processing
Error Handling Strategy
- Network Errors: Auto-retry with exponential backoff
- Rate Limits: Wait and retry automatically
- File Errors: Show user-friendly messages
- Cancellation: Clean up resources properly
This completes our multipart upload implementation! The component handles all the complexity of chunking, uploading, and error recovery while providing a clean interface for users.
Step 3: Bringing It All Together
Now letโs combine our PhotoGrid and ImageUploader components into a complete vault page that demonstrates the full user experience.
Creating the Vault Page
Hereโs how all our components work together in a production-ready vault interface:
The Main Vault Component
Key Architecture Decisions
Layout & UX
- Responsive Grid: 3-column layout on desktop, single column on mobile
- Progressive Enhancement: Works without JavaScript for basic functionality
- Clear Visual Hierarchy: Existing images prominent, upload panel secondary
State Management
- Centralized Auth: Single source of truth for user authentication
- Optimistic Updates: UI updates immediately, syncs with server
- Error Boundaries: Graceful handling of authentication and network failures
Performance Considerations
- Lazy Loading: Images load as needed
- Suspense Boundaries: Non-blocking loading states
- Efficient Re-renders: Minimal re-renders through proper key usage
Testing Your Implementation
Now that you have a complete multipart upload system, hereโs how to test it thoroughly:
1. Basic Functionality Tests
Photo Grid Loading
- Empty vault state displays correctly
- Loading spinner appears during API calls
- Error message shows with retry button for network failures
- Images display properly with hover effects
- โLoad Moreโ button works for pagination
File Upload Tests
- File picker opens and accepts image files
- Multiple file selection works correctly
- File validation rejects invalid types and sizes
- Progress tracking updates in real-time
- Completed uploads refresh the photo grid
2. Multipart Upload Flow Testing
Small Files (< 10MB)
- Should upload in single chunk
- Progress should jump quickly to 100%
- Processing state should appear briefly
Large Files (> 10MB)
- Should split into multiple chunks
- Progress should increase gradually
- Each chunk upload should be visible in network tab
Very Large Files (> 100MB)
- Should handle many chunks without browser freezing
- Memory usage should remain stable
- Upload should complete successfully
3. Error Scenario Testing
Network Interruptions
- Disconnect internet during upload
- Should show retry button when reconnected
- Auto-retry should work for retryable errors
Authentication Failures
- Use expired or invalid OAuth token
- Should show authentication error message
- Should redirect to login when appropriate
Rate Limiting
- Upload many files simultaneously
- Should handle 429 responses gracefully
- Should implement exponential backoff
File Size Limits
- Try uploading files over 1GB
- Should show clear error message before upload starts
4. User Experience Testing
Visual Feedback
- All loading states have appropriate spinners
- Error states have clear messaging
- Success states provide confirmation
Responsive Design
- Grid layout adapts to different screen sizes
- Upload interface works on mobile devices
- Touch interactions work properly
Accessibility
- Keyboard navigation works throughout
- Screen readers can announce upload progress
- Color contrast meets accessibility standards
Advanced Features
Once youโve mastered the basic multipart upload, consider adding these enhancements:
Upload Queue Management
Upload Analytics
Drag and Drop Enhancement
Performance Optimizations
1. Dynamic Chunk Sizing
2. Connection Pooling
3. Image Preprocessing
Security Considerations
1. Client-Side Validation is Not Enough
2. Secure Token Handling
3. Content Security Policy
Troubleshooting Common Issues
Upload Fails Immediately
- Check OAuth token: Ensure
read:mediaandwrite:mediascopes - Verify file type: Only images are supported in this implementation
- Check file size: Must be under 1GB limit
- Network connectivity: Ensure API endpoint is reachable
Progress Bar Stuck at 0%
- Check initiate response: Verify mediaUuid and uploadId are received
- Check part URLs: Ensure signed URLs are being generated
- Network issues: Temporary connectivity problems
Images Not Appearing After Upload
- Processing time: Large images may take minutes to process
- Status checking: Verify images reach โreadyโ status
- Variant generation: Thumbnails must be created before display
- Refresh timing: Grid should refresh after successful upload
Memory Issues with Large Files
- Chunk size: Reduce from 10MB to 5MB for slower devices
- Concurrency: Reduce MAX_CONCURRENT_UPLOADS from 3 to 2
- File cleanup: Ensure File objects are garbage collected
Next Steps
This tutorial provides a solid foundation for multipart uploads. Consider extending with:
- Resume Capability: Store upload state in localStorage
- Background Uploads: Continue uploads when tab is hidden
- Bulk Operations: Select and delete multiple images
- Advanced Metadata: EXIF data extraction and display
- Image Editing: Basic crop/rotate before upload
Production Checklist
Before deploying your multipart upload vault to production:
- OAuth tokens stored securely - Never expose tokens client-side
- Error tracking implemented - Use Sentry, LogRocket, or similar
- Upload limits configured - File size, count, and rate limits
- User feedback for all states - Loading, error, and success messaging
- Mobile-responsive design tested - Upload works on all devices
- Accessibility features added - Screen reader support, keyboard navigation
- Rate limiting handled gracefully - Exponential backoff, user messaging
- Image optimization implemented - Client-side compression where appropriate
- Content security policies configured - CSP headers to prevent XSS
- Performance monitoring - Track upload success rates and timing
- Comprehensive error logging - Log all upload failures for debugging
- Backup recovery plan - Handle corrupted uploads gracefully
This tutorial demonstrates a complete multipart upload implementation using OAuth Bearer tokens and modern React patterns. For questions about specific API endpoints, see the API Reference.