Skip to main content

CUDA & Docker on Mac — Technical Reference Guide

CUDA Development · Docker Workflows · GPU Memory Internals · July 2026

Table of Contents​

  1. CUDA on Mac with Docker
  2. GPU on Raspberry Pi
  3. Free Cloud GPU Servers for CUDA
  4. VS Code + Google Colab Integration
  5. Running CUDA C++ in a Notebook
  6. Managed vs Explicit GPU Memory
  7. How Managed Memory Works Internally
  8. Docker File Transfer on Mac (External Volume)

1. CUDA on Mac with Docker​

macOS has no NVIDIA GPU support — CUDA cannot run natively. Docker can compile CUDA C++ code using NVIDIA's official images, but GPU execution requires a Linux machine with an NVIDIA GPU.

1.1 Compile-Only Dockerfile​

FROM nvidia/cuda:12.5.0-devel-ubuntu22.04

RUN apt-get update && apt-get install -y \
    build-essential cmake && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY . .

RUN nvcc -arch=sm_86 -o my_program main.cu

Replace sm_86 with your target GPU's compute capability (sm_80 = A100, sm_75 = RTX 2080).

1.2 Build & Extract Binary​

docker build -t cuda-builder .
docker create --name temp-container cuda-builder
docker cp temp-container:/app/my_program ./my_program
docker rm temp-container

1.3 Run on a Remote Linux GPU Server​

docker run --gpus all --rm \
  -v $(pwd):/app \
  nvidia/cuda:12.5.0-devel-ubuntu22.04 \
  bash -c "cd /app && nvcc -o my_program main.cu && ./my_program"

1.4 Summary​

GoalOn Mac with DockerSolution
Compile .cu files✅ Yesnvidia/cuda:*-devel image
Run GPU code locally❌ NoNeed Linux + NVIDIA machine
Test logic without GPU⚠️ LimitedCUDA emulator (CPU-based)

2. GPU on Raspberry Pi​

Raspberry Pi has a built-in Broadcom GPU, but CUDA is NVIDIA-only and is not supported. Options depend on Pi model.

2.1 Built-in GPU Capabilities​

  • OpenGL ES / Vulkan — well supported on Pi 4/5
  • OpenCL — available via VC4CL on Pi 1–3; not supported on Pi 5 (BCM2712)
  • CUDA — not supported on any built-in VideoCore GPU

2.2 Discrete GPU via PCIe (Pi 5 / CM5)​

GPU BrandStatusNotes
AMD✅ Works well15-line kernel patch; install firmware-amd-graphics
NVIDIA⚠️ ExperimentalOpen GPU kernel module patches required
Intel Arc⚠️ PartialPatch pending upstream

3. Free Cloud GPU Servers for CUDA​

PlatformGPUFree Hours/WeekSession LimitCUDA C++
Google ColabT4 16GB~15–30 hrs12 hrs✅ Yes
KaggleT4 / P10030 hrs9 hrs✅ Yes
SageMaker Studio LabT4~28 hrs4 hrs✅ Yes
Lightning.aiT4Limited creditsFlexible✅ Yes

Recommendation: Use Kaggle for the most reliable free quota, or Google Colab for easiest CUDA C++ setup with nvcc4jupyter.

4. VS Code + Google Colab Integration​

Google officially launched the Colab VS Code extension on November 13, 2025. The browser Colab session and VS Code session are completely independent — changing the runtime in the browser does NOT affect VS Code.

4.1 Connect to T4 GPU from VS Code​

  1. Click kernel name (top-right) → "Select Another Kernel..."
  2. Choose "Google Colab" → "New Colab Server"
  3. Select GPU → T4
  4. Verify with !nvidia-smi

⚠️ GPU availability is not guaranteed during peak hours (9 AM–6 PM Pacific). Try at off-peak times if T4 is unavailable.

4.2 Disconnect When Done​

To preserve your free GPU quota, always disconnect after use:

Cmd+Shift+P → "Colab: Remove Server"

5. Running CUDA C++ in a Notebook​

Option A — nvcc4jupyter Cell Magic​

⚠️ The %load_ext and %%cuda magic must be in separate cells — this is the most common mistake.

# Cell 1
!pip install nvcc4jupyter

# Cell 2
%load_ext nvcc4jupyter

# Cell 3
%%cuda
#include <stdio.h>
__global__ void hello() {
    printf("Hello from thread %d!\n", threadIdx.x);
}
int main() {
    hello<<<1, 8>>>();
    cudaDeviceSynchronize();
    return 0;
}

Option B — Shell Compilation (More Reliable)​

# Write source file
with open('hello.cu', 'w') as f:
    f.write(cuda_code)

# Compile and run
!nvcc -o hello hello.cu && ./hello
MethodProsCons
%%cuda magicClean notebook cellsFragile, version-sensitive
!nvcc shellAlways works, no depsSlightly more verbose

6. Managed vs Explicit GPU Memory​

6.1 Why Managed Memory (cudaMallocManaged) is Slower​

CauseDetail
Page fault handlingData migrates on-demand per 4KB page, with OS interrupt overhead each time
Migration granularityMany small transfers vs. one bulk DMA with explicit memory
TLB invalidationsBoth CPU and GPU TLBs must be flushed on each page migration
Runtime bookkeepingDriver tracks ownership of every page across both address spaces

6.2 Is Explicit Still Faster Even Counting cudaMemcpy Overhead?​

Yes — a single bulk DMA transfer saturates the PCIe bus efficiently. Page-fault migration does many small transfers with per-transfer OS overhead, which is much worse even if total bytes are the same.

Explicit:  one bulk DMA → PCIe fully saturated for the duration
Managed:   262,144 × (DMA + unmap + TLB flush + remap) for a 1 GB allocation

Exception: managed memory can win when access patterns are sparse — if the kernel only touches a fraction of a large allocation, managed memory only migrates those pages.

6.3 Concurrency Model​

HardwareBehavior
Pascal and olderAll pages migrated to GPU upfront on kernel launch; CPU locked out entirely
Volta+ with NVLinkTrue concurrency — CPU and GPU can access different pages simultaneously
Same page, both sidesAlways a race condition — programmer's responsibility to avoid

6.4 cudaMemPrefetchAsync — Best of Both Worlds​

// Migrate pages to GPU before kernel launch (async, no faults during kernel)
cudaMemPrefetchAsync(ptr, size, deviceId, stream);

// Kernel runs without page faults — data already in VRAM
kernel<<<grid, block, 0, stream>>>(ptr);

6.5 Rule of Thumb​

ScenarioRecommendation
Dense, predictable accessExplicit (cudaMalloc + cudaMemcpy)
Sparse or unpredictable accessManaged (cudaMallocManaged)
Prototyping / correctness testingManaged (simpler code)
Production / performance criticalExplicit

7. How Managed Memory Works Internally​

7.1 Unified Virtual Address Space​

Both CPU and GPU share the same virtual address for a managed allocation. The pointer from cudaMallocManaged is valid on both sides, but physical data can only live in one location at a time.

7.2 Migrate = Copy + Unmap + Free​

Migration is NOT a simple pointer swap. Full sequence when a GPU kernel touches a page currently on the CPU:

Step 1: DMA COPY    — bits transferred CPU RAM → GPU VRAM over PCIe
                      (~10–20 GB/s on PCIe 4.0; this is the slow step)

Step 2: UNMAP CPU   — CPU page table entry marked "not present"
                      CPU TLB flushed; CPU segfaults if it touches the address

Step 3: MAP GPU     — GPU page table updated to point to new VRAM frame
                      GPU TLB updated; faulting warp resumes execution

Step 4: FREE frame  — CPU physical frame returned to OS allocator
TermBoth sides valid?Source destroyed?
Copy✅ Yes❌ No
Migrate❌ No (briefly same bits, only dest mapped)✅ Yes (unmapped + freed)

7.3 Why This Is Slower Than cudaMemcpy​

cudaMemcpy (explicit):
  One bulk DMA — PCIe fully saturated for the duration
  No page table manipulation, no TLB flushes

cudaMallocManaged (page fault driven):
  Per 4KB page: DMA copy + unmap + TLB flush + remap + warp resume
  1 GB allocation = 262,144 pages × overhead per page
  PCIe used in small, unoptimized bursts

8. Docker File Transfer on Mac (External Volume)​

8.1 The Problem​

Docker Desktop on Mac runs Linux in a VM and can only access paths explicitly shared with that VM. External volumes (/Volumes/...) are not accessible by default, causing docker cp to fail:

Can't add file /Volumes/.../file.zip to tar: io: read/write on closed pipe

⚠️ Even when docker cp reports "Successfully copied", the transfer may have silently failed if the source is on an external volume. Always verify with ls after copying.

8.2 Attempted Fixes and Results​

FixResult
Docker Desktop → Resources → File Sharing → add /Volumes/...Path silently removed after clicking Apply — Docker Desktop bug
Copy file to ~/ first, then docker cpFails if home directory is also on external volume
docker cp with trailing slashCorrect syntax but doesn't fix the volume access issue

8.3 Working Solution — cat Pipe​

Stream file bytes through stdin, bypassing filesystem sharing entirely:

# Mac → Container (any file size)
cat /Volumes/MacMiniUserDisk/path/to/file.zip | \
  docker exec -i <container> sh -c 'cat > /destination/file.zip'

# Container → Mac
docker exec <container> cat /path/to/file > \
  /Volumes/MacMiniUserDisk/path/to/destination/file

This works because it pipes raw bytes through stdin/stdout — no volume mounting or filesystem sharing required.

8.4 Identifying the Right Container​

The nsenter container (named condescending_merkle in this session) is Docker Desktop's internal VM shell — not a regular container. Installing packages or copying files into it affects the Docker VM, not your build environment.

# Check which container your shell is in
hostname
# "docker-desktop" = you're in the VM shell (wrong)
# random hash      = you're in a real container (correct)
# Start a clean build container
docker run -it -d --name builder debian:latest

# Install build tools
docker exec builder apt-get update
docker exec builder apt-get install -y g++ cmake build-essential unzip nano

# Transfer files using cat pipe
cat /Volumes/MacMiniUserDisk/path/to/file.zip | \
  docker exec -i builder sh -c 'cat > /home/work/file.zip'

# Open interactive shell
docker exec -it builder bash

8.6 Common apt-get Issues in Containers​

ErrorFix
E: Unable to locate package XRun apt-get update first, then install
dpkg: error processing package (debconf frontend)Use DEBIAN_FRONTEND=noninteractive apt-get install -y
Package installs but binary not foundWrong container — check hostname; you may be in the VM shell

End of Document