|
| 1 | +# Apple Silicon Performance Guide |
| 2 | + |
| 3 | +Helicon runs on Apple Silicon (M-series) in three modes: |
| 4 | + |
| 5 | +| Component | Hardware | Notes | |
| 6 | +|-----------|----------|-------| |
| 7 | +| WarpX PIC (CPU) | P-cores via OpenMP | No Metal backend in upstream WarpX | |
| 8 | +| MLX field solver / postprocessing | Metal GPU | Biot-Savart, thrust reduction, surrogate | |
| 9 | +| warpx-metal (PIC) | Metal GPU | Native SYCL/Metal build, single precision | |
| 10 | + |
| 11 | +## Quick start: profile your machine |
| 12 | + |
| 13 | +```bash |
| 14 | +helicon perf |
| 15 | +``` |
| 16 | + |
| 17 | +Output example (M4 Pro, 48 GB): |
| 18 | + |
| 19 | +``` |
| 20 | +============================================================ |
| 21 | +Helicon Apple Silicon Performance Profile |
| 22 | +============================================================ |
| 23 | + Chip: Apple M4 Pro |
| 24 | + P-cores: 12 |
| 25 | + E-cores: 4 |
| 26 | + GPU cores: 38 |
| 27 | + Unified memory: 48 GB |
| 28 | + Mem bandwidth: 98.3 GB/s (NumPy stream-copy estimate) |
| 29 | + MLX available: True (v0.31.0) |
| 30 | + Python: 3.12.11 |
| 31 | + NumPy: 2.4.2 |
| 32 | +============================================================ |
| 33 | +
|
| 34 | +WarpX OpenMP Tuning |
| 35 | +---------------------------------------- |
| 36 | + OMP_NUM_THREADS=12 |
| 37 | + OMP_PLACES=cores |
| 38 | + OMP_PROC_BIND=close |
| 39 | + Rationale: Use only P-cores (12) — E-cores (4) are slower and create |
| 40 | + load imbalance in PIC loops. OMP_PLACES=cores pins threads to physical cores. |
| 41 | +
|
| 42 | +Shell snippet: |
| 43 | +export OMP_NUM_THREADS=12 |
| 44 | +export OMP_PLACES=cores |
| 45 | +export OMP_PROC_BIND=close |
| 46 | +``` |
| 47 | + |
| 48 | +Get a JSON-serialisable report: |
| 49 | + |
| 50 | +```bash |
| 51 | +helicon perf --json > perf_report.json |
| 52 | +``` |
| 53 | + |
| 54 | +--- |
| 55 | + |
| 56 | +## WarpX OpenMP tuning |
| 57 | + |
| 58 | +### Use only P-cores |
| 59 | + |
| 60 | +M-series chips have two core types. P-cores (performance) are 3–5× faster |
| 61 | +per thread than E-cores (efficiency) for PIC inner loops. Mixing them causes |
| 62 | +thread-barrier imbalance and degrades throughput. |
| 63 | + |
| 64 | +```bash |
| 65 | +# Recommended: pin WarpX to P-cores only |
| 66 | +export OMP_NUM_THREADS=$(helicon perf --json | python3 -c \ |
| 67 | + "import sys,json; print(json.load(sys.stdin)['openmp']['omp_num_threads'])") |
| 68 | +export OMP_PLACES=cores |
| 69 | +export OMP_PROC_BIND=close |
| 70 | +helicon run --config my_nozzle.yaml --output results/ |
| 71 | +``` |
| 72 | + |
| 73 | +### Thread binding matters |
| 74 | + |
| 75 | +`OMP_PLACES=cores` + `OMP_PROC_BIND=close` ensures each thread is pinned to a |
| 76 | +physical core and neighbouring threads share L2 cache. This matters because |
| 77 | +particle-in-cell loops are memory-bandwidth-bound — cache locality is critical. |
| 78 | + |
| 79 | +### Memory bandwidth headroom |
| 80 | + |
| 81 | +WarpX's particle push is bounded by memory bandwidth (~100 GB/s on M4 Pro). |
| 82 | +At 1M particles with 7 floats each (28 bytes/particle), a single timestep |
| 83 | +reads/writes ~56 MB. You can sustain ~1800 steps/second before memory saturation |
| 84 | +on M4 Pro with 12 P-cores. |
| 85 | + |
| 86 | +--- |
| 87 | + |
| 88 | +## MLX on Metal GPU |
| 89 | + |
| 90 | +Helicon uses MLX for every non-PIC computation: B-field solving, thrust |
| 91 | +reduction, analytical screening, surrogate inference, and Monte Carlo UQ. |
| 92 | + |
| 93 | +### Enable mlx.core.compile |
| 94 | + |
| 95 | +Compiled kernels fuse Metal dispatch calls and eliminate kernel launch overhead. |
| 96 | +Helicon enables compilation automatically where it's beneficial. To check: |
| 97 | + |
| 98 | +```python |
| 99 | +import mlx.core as mx |
| 100 | +from helicon.fields.biot_savart import _compute_mlx |
| 101 | + |
| 102 | +# This call will compile on first run (~0.5 s), then reuse the compiled graph |
| 103 | +Br, Bz, r, z = _compute_mlx(coils, grid, n_phi=64) |
| 104 | +mx.eval(Br, Bz) |
| 105 | +``` |
| 106 | + |
| 107 | +### Batch size tuning |
| 108 | + |
| 109 | +The optimal `vmap` batch size for Biot-Savart depends on GPU core count: |
| 110 | + |
| 111 | +| GPU cores | Recommended batch | |
| 112 | +|-----------|------------------| |
| 113 | +| ≥ 38 (M4 Pro/Max) | 2048 coils | |
| 114 | +| ≥ 20 (M3 Pro) | 1024 coils | |
| 115 | +| ≥ 10 (M2 Pro) | 512 coils | |
| 116 | +| < 10 | 256 coils | |
| 117 | + |
| 118 | +`helicon perf` reports the recommended batch size for your chip. |
| 119 | + |
| 120 | +### Lazy evaluation |
| 121 | + |
| 122 | +MLX uses lazy evaluation — operations are queued and executed only when |
| 123 | +`mx.eval()` is called. For large scans: |
| 124 | + |
| 125 | +```python |
| 126 | +import mlx.core as mx |
| 127 | + |
| 128 | +results = [] |
| 129 | +for i, candidate in enumerate(candidates): |
| 130 | + r = analytical_screen(candidate) |
| 131 | + results.append(r) |
| 132 | + if i % 1024 == 0: |
| 133 | + mx.eval(*results) # flush every 1024 to prevent queue overflow |
| 134 | + results = [] |
| 135 | +mx.eval(*results) |
| 136 | +``` |
| 137 | + |
| 138 | +--- |
| 139 | + |
| 140 | +## Unified memory management |
| 141 | + |
| 142 | +Apple Silicon has a single unified memory pool shared between CPU (WarpX) and |
| 143 | +GPU (MLX). Simultaneous WarpX + MLX pressure can cause memory stalls. |
| 144 | + |
| 145 | +### Capacity estimates |
| 146 | + |
| 147 | +`helicon perf` computes conservative capacity estimates based on unified memory: |
| 148 | + |
| 149 | +- **Max particles**: `0.70 × memory_GB × 1e9 / 28 bytes` (7 floats/particle) |
| 150 | +- **Recommended**: 60% of max (leaves headroom for diagnostics + Python heap) |
| 151 | +- **Max grid** (nz×nr): estimated from field-array footprint (7 fields × float32) |
| 152 | + |
| 153 | +Example for 48 GB M4 Pro: |
| 154 | + |
| 155 | +| | Value | |
| 156 | +|--|-------| |
| 157 | +| Max particles | ~1200M | |
| 158 | +| Recommended | ~720M | |
| 159 | +| Max grid (nz×nr) | ~8192×4096 | |
| 160 | + |
| 161 | +### Reduce diagnostic footprint |
| 162 | + |
| 163 | +Full diagnostic mode (analysis) writes particle dumps every 5000 steps. |
| 164 | +For parameter scans use `scan` mode: |
| 165 | + |
| 166 | +```yaml |
| 167 | +diagnostics: |
| 168 | + mode: scan # no particle dumps; field dumps only |
| 169 | + field_dump_interval: 500 |
| 170 | +``` |
| 171 | +
|
| 172 | +This cuts storage from ~5 GB to ~50 MB per run and reduces unified memory |
| 173 | +pressure during postprocessing. |
| 174 | +
|
| 175 | +### Keep checkpoints off |
| 176 | +
|
| 177 | +```yaml |
| 178 | +keep_checkpoints: false # default; saves 50–80% disk space |
| 179 | +``` |
| 180 | +
|
| 181 | +Enable only for multi-day runs where restart capability is needed. |
| 182 | +
|
| 183 | +--- |
| 184 | +
|
| 185 | +## AMR (Adaptive Mesh Refinement) |
| 186 | +
|
| 187 | +WarpX supports native AMR to resolve the narrow electron demagnetization layer |
| 188 | +(detachment front) without paying the cost of uniform fine grid everywhere. |
| 189 | +
|
| 190 | +Enable in the YAML config: |
| 191 | +
|
| 192 | +```yaml |
| 193 | +nozzle: |
| 194 | + resolution: |
| 195 | + nz: 512 |
| 196 | + nr: 256 |
| 197 | + amr_max_level: 1 # one level of factor-2 refinement |
| 198 | + amr_ref_ratio: 2 # refinement ratio |
| 199 | + amr_regrid_int: 10 # regrid every 10 steps |
| 200 | +``` |
| 201 | +
|
| 202 | +The WarpX input file will include: |
| 203 | +
|
| 204 | +``` |
| 205 | +amr.max_level = 1 |
| 206 | +amr.ref_ratio = 2 |
| 207 | +amr.regrid_int = 10 |
| 208 | +amr.blocking_factor = 8 |
| 209 | +amr.max_grid_size = 128 |
| 210 | +warpx.refine_plasma = 1 # refine where plasma density is highest |
| 211 | +amr.n_error_buf = 2 |
| 212 | +``` |
| 213 | + |
| 214 | +**Expected benefit:** ~40–50% wall-time reduction vs. a uniformly doubled |
| 215 | +grid for the same detachment-front resolution, as the fine region is typically |
| 216 | +<20% of the domain area. |
| 217 | + |
| 218 | +**Note:** AMR is not supported by warpx-metal (the SYCL/Metal build uses a |
| 219 | +uniform grid). AMR applies only to CPU OpenMP and CUDA backends. |
| 220 | + |
| 221 | +--- |
| 222 | + |
| 223 | +## warpx-metal: PIC on Metal GPU |
| 224 | + |
| 225 | +For full PIC on the Metal GPU, build [warpx-metal](https://github.com/lulzx/warpx-metal): |
| 226 | + |
| 227 | +```bash |
| 228 | +git clone https://github.com/lulzx/warpx-metal ../warpx-metal |
| 229 | +cd ../warpx-metal |
| 230 | +./scripts/00-install-deps.sh |
| 231 | +./scripts/01-build-adaptivecpp.sh |
| 232 | +./scripts/05-build-warpx.sh |
| 233 | +``` |
| 234 | + |
| 235 | +Helicon auto-detects the build at `../warpx-metal` or `~/work/warpx-metal`. |
| 236 | + |
| 237 | +**First-run JIT:** AdaptiveCpp compiles LLVM IR → Metal shaders on first use |
| 238 | +(~2–10 s/step for the first ~200 steps). Subsequent runs reuse the JIT cache |
| 239 | +at `~/.acpp/apps/global/jit-cache/` and reach ~1 ms/step. |
| 240 | + |
| 241 | +**Limitations:** single precision (FP32), FDTD only (no PSATD), 2D Cartesian |
| 242 | +only (no RZ), periodic boundaries only (PML triggers a Metal JIT bug for |
| 243 | +D⁺/e⁻ mass-ratio plasmas). |
| 244 | + |
| 245 | +```bash |
| 246 | +# Check Metal detection |
| 247 | +helicon doctor |
| 248 | + |
| 249 | +# Run with Metal backend (auto-selected when warpx-metal is found) |
| 250 | +helicon run --preset sunbird --output results/ |
| 251 | +``` |
| 252 | + |
| 253 | +Override the step cap: |
| 254 | + |
| 255 | +```bash |
| 256 | +HELICON_METAL_MAX_STEP=2000 helicon run --preset sunbird --output results/ |
| 257 | +``` |
0 commit comments