A technical deep dive into the brand new deep studying library MLX

In December 2023, Apple launched their new MLX deep studying framework, an array framework for machine studying on Apple silicon, developed by their machine studying analysis workforce. This tutorial will discover the framework and show deploying the Mistral-7B mannequin domestically on a MacBook Professional (MBP). We’ll arrange a neighborhood chat interface to work together with the deployed mannequin and take a look at its inference efficiency by way of tokens generated per second. Moreover, we’ll delve into the MLX API to grasp the out there levers for altering the mannequin’s behaviour and influencing the generated textual content.

As ordinary, the code is accessible in a public GitHub repository: https://github.com/marshmellow77/mlx-deep-dive

Apple’s new machine studying framework, MLX, gives notable benefits over different deep studying frameworks with its unified reminiscence structure for machine studying on Apple silicon. Not like conventional frameworks resembling PyTorch and Jax, which require expensive knowledge copying between CPU and GPU, MLX maintains knowledge in shared reminiscence accessible to each. This design eliminates the overhead of knowledge transfers, facilitating sooner execution, notably with the massive datasets widespread in machine studying. For advanced ML duties on Apple units, MLX’s shared reminiscence structure might result in vital speed-ups. This characteristic makes MLX extremely related for builders seeking to run fashions on-device, resembling on iPhones.

With Apple’s experience in silicon design, MLX hints on the thrilling capabilities that may very well be built-in into their chips for future on-device AI purposes. The potential of MLX to speed up and streamline ML duties on Apple platforms makes it a framework builders ought to carry on their radar.

Earlier than deploying the mannequin, some setup is required. Firstly, it’s important to put in sure libraries. Bear in mind to create a digital setting earlier than continuing with the installations:

pip set up mlx-lm