PyTorch on Apple Silicon | Machine Learning | M1 Max/Ultra vs nVidia

Wow! Looks like I need to upgrade to an M1 Ultra, turns out 16.44 mins is just enough time to make and eat a Vegemite sandwich!

Hahaha, excellent video Alex, looking forward to the next one :)

Using c++ makes life faster

Hi Alex, can u check the results with the latest stable version of torch?

So this "mps" is using Metal API, but will Pytorch support Neural Engine? Is Neural Engine lie dormant in Apple Silicon SoC?

What about YoloV8? :)

Macbook Pro 16 M2 Max 32GB 38 GPU cores result (PyTorch 2.0.0):

torch 2.0.0
device mps
Epoch: 001/001 | Batch 0000/1406 | Loss: 2.6346
Epoch: 001/001 | Batch 0100/1406 | Loss: 2.2348
Epoch: 001/001 | Batch 0200/1406 | Loss: 2.1773
Epoch: 001/001 | Batch 0300/1406 | Loss: 2.3495
Epoch: 001/001 | Batch 0400/1406 | Loss: 2.3165
Epoch: 001/001 | Batch 0500/1406 | Loss: 2.1477
Epoch: 001/001 | Batch 0600/1406 | Loss: 2.0689
Epoch: 001/001 | Batch 0700/1406 | Loss: 2.0424
Epoch: 001/001 | Batch 0800/1406 | Loss: 1.9650
Epoch: 001/001 | Batch 0900/1406 | Loss: 1.9270
Epoch: 001/001 | Batch 1000/1406 | Loss: 1.8402
Epoch: 001/001 | Batch 1100/1406 | Loss: 1.8375
Epoch: 001/001 | Batch 1200/1406 | Loss: 1.8020
Epoch: 001/001 | Batch 1300/1406 | Loss: 1.9095
Epoch: 001/001 | Batch 1400/1406 | Loss: 2.0477
Time / epoch without evaluation: 9.76 min
Epoch: 001/001 | Train: 25.62% | Validation: 25.72% | Best Validation (Ep. 001): 25.72%
Time elapsed: 12.44 min
Total Training Time: 12.44 min
Test accuracy 26.20%
Total Time: 13.16 min

Macbook Pro 16 M2 Max 32GB 38 GPU cores result (PyTorch 2.0.0):

torch 2.0.0
device mps
Epoch: 001/001 | Batch 0000/1406 | Loss: 2.6346
Epoch: 001/001 | Batch 0100/1406 | Loss: 2.2348
Epoch: 001/001 | Batch 0200/1406 | Loss: 2.1773
Epoch: 001/001 | Batch 0300/1406 | Loss: 2.3495
Epoch: 001/001 | Batch 0400/1406 | Loss: 2.3165
Epoch: 001/001 | Batch 0500/1406 | Loss: 2.1477
Epoch: 001/001 | Batch 0600/1406 | Loss: 2.0689
Epoch: 001/001 | Batch 0700/1406 | Loss: 2.0424
Epoch: 001/001 | Batch 0800/1406 | Loss: 1.9650
Epoch: 001/001 | Batch 0900/1406 | Loss: 1.9270
Epoch: 001/001 | Batch 1000/1406 | Loss: 1.8402
Epoch: 001/001 | Batch 1100/1406 | Loss: 1.8375
Epoch: 001/001 | Batch 1200/1406 | Loss: 1.8020
Epoch: 001/001 | Batch 1300/1406 | Loss: 1.9095
Epoch: 001/001 | Batch 1400/1406 | Loss: 2.0477
Time / epoch without evaluation: 9.76 min
Epoch: 001/001 | Train: 25.62% | Validation: 25.72% | Best Validation (Ep. 001): 25.72%
Time elapsed: 12.44 min
Total Training Time: 12.44 min
Test accuracy 26.20%
Total Time: 13.16 min

There is a bit of a problem with your conclusion/results "The Titan" is useless in telling us anything about the Nvidia card used as there have been 8 "Titan" cards, with the most recent being released in 2018 (Titan RTX), and of course its going to be expensive buying that new from a retailer (Amazon, Newegg, Etc.) because it's not in production anymore. Used they have sold for around 7-800$ on Ebay or new for around $1200: however the cheapest you can get an M1 Ultra is around $4000 (as of this writing). Obviously a Mac Studio is a lot more than just a single GPU, however you can buy a lot of computer for $2800. Also you say this test should focus on the difference between CPU, and GPU; however your video title is "M1 Max/Ultra vs nVidia".

The irony of a Mac guy making a comment about the price of an nvidia titan card.

The nice thing about this is that it's easier to get a machine with tons of RAM than tons of VRAM...though I don't know if apple will let you upgrade your RAM for apple silicon any time soon.

Is this possible to run a docker instance that takes advantage of pytorch on apple metal?

The training use the GPU core or the Neural Engine?

Nice video and script. 3080 laptop, WSL PyTorch 1.13.0: total time 10.54min -> laptop 3080 is largely the same as a desktop 1080ti.

M1 Max 32 core GPU on 16 inch macbook pro, I got Total Time: 19.02 min. PyTorch 1.13.0 stable. My only question is Python 3.10 seems to be intel version, is it an option to install apple silicon native python? It may further improve the performance.

Silly question, which monitor support are you using, looks pretty neat

What an awesome videocasting steup you have right there! I quite envy it! Haha! 😅😅😅

That was amazing thanks guys! I definitely need one of those for my job :)

Nice!

So apple has produced some great cpus here , especially the unified memory. But don't believe the hype for ai workloads. A rtx 3090 with a decent Intel cpu will blow away any apple pc. For laptops, apple is the most power efficient solution.

Thanks Alex for the great content. Did you check it again with the stable release of PyTorch 1.12 to see if there are performance improvements?