Arbitrary Style Transfer in the Browser
A client-side Next.js web application performing arbitrary image style transfer entirely in the browser via TensorFlow.js and WebGL shaders, with zero server-side image processing.
Timeline
Nov 2025
Role
Solo — web pipeline, model optimization, UI/UX
Status
CompletedTechnology Stack
Key Challenges
- Serving and lazy-loading 56 MB of sharded model weights without page freezing
- Managing WebGL GPU memory lifecycle manually to prevent browser tab crashes
- Implementing smooth style interpolation between two arbitrary styles in bottleneck space
Key Learnings
- Client-side inference provides complete user privacy while eliminating server GPU costs
- Explicit `tf.tidy()` and `tensor.dispose()` calls are essential for long-running browser graphics sessions
- Exposing model quality vs. speed tiers gives users control over mobile and desktop performance
Overview
Traditional neural style transfer requires dedicated GPU server backends where users must upload personal photos.
This project executes arbitrary image style transfer entirely inside the client's browser via TensorFlow.js and WebGL acceleration. Images never leave the user's device, eliminating cloud inference bills and guaranteeing privacy.
Architecture: Two-Stage Neural Pipeline
Instead of retraining a network for every style image, the pipeline separates stylization into two distinct feed-forward networks (Ghiasi et al., 2017):
- Style Predictor Network: Encodes any reference style image into a compact 100-dimensional style bottleneck vector.
- Transform Network: Takes the content image alongside the style bottleneck and synthesizes the final stylized output in a single forward pass.
Because the bottleneck is a dense numerical vector, users can blend two different styles in real time by interpolating bottleneck vectors before applying the transformation.
Model Tiers & Lazy Loading
To balance performance across mobile devices and desktop GPUs, the application provides swappable model tiers loaded on demand:
| Tier | Models | Combined Size | Target Devices |
|---|---|---|---|
| Fast (Default) | MobileNet + Separable Transformer | ~12 MB | Mobile & Laptops |
| High Quality | Inception-v3 + Standard Transformer | ~44 MB | Desktop Workstations |
Model weights are served in sharded binary chunks with HTTP caching, ensuring instant subsequent loads.
Memory Management
Because WebGL textures are allocated directly in GPU VRAM without JavaScript garbage collection, every intermediate computation is wrapped in tf.tidy() scopes and explicitly disposed via tensor.dispose().
Tech Stack Summary
- Frontend: Next.js 15, React 18, Tailwind CSS, shadcn/ui
- Machine Learning: TensorFlow.js, WebGL Backend
- Key Features: Client-Side Inference, Multi-Style Blending, Web Workers