Metal 4 / M5 support (Tensor APIs to program GPU Neural Accelerators)

[reneleonhardt]

Feature description

Has the current Metal 4 already been implemented to support the new Tensor APIs for Neural Accelerators on Apple Silicon M5?

Feature motivation

https://www.macrumors.com/guide/m4-vs-m5-chip/

• 4×+ peak GPU compute performance for AI
• 3.6× faster time to first token (LLM)

(Optional) Suggest a Solution

https://developer.apple.com/metal/whats-new/

Metal 4 machine learning

Tensors are now a native resource type you can use to operate on data.

[nathanielsimard]

Is there some documentation on how to program those tensor cores? We are already using the matrix multiplication instruction (8,8,8), but unsure if there is a new one for the new tensor cores.

[reneleonhardt]

Apple's documentation is not very well structured in my eyes, the landing page seems to appeal only to game developers, the overview doesn't mention Tensor cores but at least the new MTL4MachineLearningCommandEncoder.
https://developer.apple.com/documentation/metal/understanding-the-metal-4-core-api

The CoreML overview mentions withMLTensorComputePolicy().
https://developer.apple.com/documentation/CoreML

At least "Machine-learning passes" is all about Metal 4 to run CoreML models efficiently.
https://developer.apple.com/documentation/metal/machine-learning-passes

Perplexity generates example code for the new device.makeTensor(MTLCTensorDescriptor).
https://www.perplexity.ai/search/does-apple-offer-documentation-agmxKYCqTe2gw9qNoHHBaQ#1

Accelerate machine learning with Metal

Learn how to accelerate your machine learning transformer models with new features in Metal Performance Shaders Graph.

https://wwdcnotes.com/documentation/wwdcnotes/wwdc24-10218-accelerate-machine-learning-with-metal/

https://wwdcnotes.com/documentation/wwdcnotes/wwdc25-262-combine-metal-4-machine-learning-and-graphics/

Metal Tensors

Multi-dimensional data containers that extend beyond 2D:

• Native API integration
• Metal Shading Language support
• Complex indexing handled automatically

https://dev.to/arshtechpro/wwdc-2025-discover-metal-4-23f2

A Summary of the WWDC25 Group Lab - Machine Learning and AI Frameworks

https://developer.apple.com/forums/topics/machine-learning-and-ai