Discover ways to use OnnxStream to generate photos with Secure Diffusion XL Turbo on the Raspberry Pi!

In my final article, I shared the way to run giant language fashions and imaginative and prescient language fashions on the Raspberry Pi. This time round, as an alternative of LLMs and VLMs, we will run a picture era mannequin — Secure Diffusion XL (SDXL) Turbo — on the Raspberry Pi 5. It’s one other not possible sounding feat, however open-source wonders do exist, and working a SDXL Turbo mannequin on a really useful resource constrained surroundings is one in all them.

OnnxStream is an open-source challenge created by Vito Plantamura with the unique intention of working Secure Diffusion 1.5 (SD1.5) on a Raspberry Pi Zero 2 by minimising reminiscence consumption as a lot as attainable, albeit at the price of elevated inference latency/throughput.

On the time of writing, it has expanded to help not solely Secure Diffusion 1.5 but additionally Secure Diffusion XL 1.0 Base (SDXL) and Secure Diffusion XL Turbo 1.0. I gained’t go into element about how precisely this wonderful feat is being achieved because the GitHub repository already explains it very nicely.

As a substitute, let’s simply soar proper into getting it working.

All you want is the next:

• Raspberry Pi 5 — Or Raspberry Pi 4 or some other Raspberry Pi, simply count on it to be slower
• SD Card — Minimally 16GB, with Raspbian or some Linux distro already setup. The SDXL Turbo weights are round 8GB.
• An web connection

The directions listed below are from the GitHub repository, however I’m breaking it down and explaining it a bit extra.

1. Constructing XNNPack

First, we have now to put in XNNPack, which is a library from Google that gives “high-efficiency floating-point neural community inference operators”. However we will’t simply get the newest model in case of any breaking adjustments. As a substitute, we will get the model that the OnnxStream creator has verified to be working on the time of writing. In a terminal, run:

git clone https://github.com/google/XNNPACK.git
cd XNNPACK
git checkout 579de32260742a24166ecd13213d2e60af862675
mkdir construct
cd construct
cmake -DXNNPACK_BUILD_TESTS=OFF -DXNNPACK_BUILD_BENCHMARKS=OFF ..
cmake --build . --config Launch

XNNPack will take a few minutes to construct. Go get espresso or one thing.

2. Constructing OnnxStream

Subsequent, we have now to construct OnnxStream. In a terminal, run:

git clone https://github.com/vitoplantamura/OnnxStream.git
cd OnnxStream
cd src
mkdir construct
cd construct
cmake -DMAX_SPEED=ON -DXNNPACK_DIR=<DIRECTORY_WHERE_XNNPACK_WAS_CLONED> ..
cmake --build . --config Launch

Make sure that to interchange <DIRECTORY_WHERE_XNNPACK_WAS_CLONED> with the trail at which XNNPack was cloned to (not the construct folder). In my case, it was at /house/admin/XNNPACK/.

3. Downloading Mannequin Weights

Now, we have to obtain the mannequin weights for SDXL Turbo. In a terminal, run:

git lfs set up
git clone --depth=1 https://huggingface.co/AeroX2/stable-diffusion-xl-turbo-1.0-onnxstream

When you have not put in git-lfs but, achieve this first. This can take even longer than the step earlier than because the mannequin weights are fairly huge. Go get lunch!

You too can run the opposite two fashions supported — Secure Diffusion 1.5 and Secure Diffusion XL 1.0 Base by downloading their weights from the hyperlinks supplied in OnnxStream’s GitHub repository. Make sure that your SD card has sufficient house if you’re downloading all these fashions!

As soon as achieved, that’s it! We’re able to generate photos on a Raspberry Pi!

To generate photos, run the code block beneath:

cd ~/OnnxStream/src/construct/
./sd --turbo --models-path /house/admin/stable-diffusion-xl-turbo-1.0-onnxstream --prompt "An astronaut using a horse on Mars" --steps 1 --output astronaut.png

Substitute the immediate with what you wish to generate. I’m simply utilizing the go-to basic astronaut immediate right here. I set steps to simply 1 as SDXL Turbo doesn’t want many steps to generate a handsome picture.

There are different arguments you possibly can move too, reminiscent of — neg-prompt for unfavorable prompts (SDXL Turbo doesn’t help unfavorable prompts however you should use it for the opposite two fashions), — steps to set the variety of generative steps and — seed to set the random seed.

The arguments required will change in accordance with the kind of mannequin used, so please check out OnnxStream’s GitHub repository for the total record of arguments to move if you happen to’re utilizing one thing aside from SDXL Turbo.

As you possibly can see within the picture above, on the Raspberry Pi 5, every diffusion step takes round 1 minute, and in complete with pre-processing and decoding, it round 3 minutes to generate a single picture.

Right here’s a comparability and development of the identical immediate with the identical seed from step 1 to 10. You may see that even with a single step with refinement, the generated picture is already actually well-done. That is in distinction to SDXL or SD1.5 which requires fairly a couple of steps to achieve that high quality.

With it taking round not less than a few minutes to generate a picture, the query of use circumstances for it comes begging.

The obvious and enjoyable use case to me is an ever-changing photograph body that can generate a brand new picture each couple of minutes. There’s truly a challenge alongside this tangent that makes use of OnnxStream, by rvdveen on GitHub, the place he makes use of OnnxStream on a Raspberry Pi Zero 2 W to generate photos for information articles and exhibits it on a photograph body utilizing an e-ink show. It takes round 5 hours to generate a picture on the Pi Zero 2 W with what must be SD1.5, however hey it’s not such as you want a photograph body to be altering what it’s exhibiting in real-time.

Or possibly you simply need your personal domestically hosted picture generator which may produce respectable high quality photos whereas not hogging any main compute gadgets in your community.

Have enjoyable enjoying round with SDXL Turbo on the Raspberry Pi!

Disclaimer: I’ve no affiliation with OnnxStream or StabilityAI. All views and opinions are my very own and don’t symbolize any organisation.