Constructing an Open Supply “ChatGPT” App on iPhone Utilizing Ollama and Google Colab Free T4 GPU

Tired of paying pricey subscriptions or cautious of sharing your private knowledge with OpenAI?

What if there have been free and safer alternate options utilizing very succesful open supply fashions?

For those who’re intrigued, then this information is for you. Let’s construct collectively our personal “ChatGPT,” powered by probably the most succesful open supply fashions, proper in your iPhone!

On the backend, we’ll leverage Ollama and Google Colab’s free T4 GPU to serve the LLMs. For the frontend, we’ll make use of Enchanted, a sublime open supply iOS app, to work together with fashions equivalent to Llama 2, Mistral, Phi-2, and extra.

By the tip of this information, you’ll have a robust AI at your fingertips — with out spending a dime. And one of the best half? You may simply swap among the many finest open supply fashions in response to your wants!

Prepared? Let’s dive in!

To construct our open supply “ChatGPT”, we’ll use the next key parts :

• Google Colab pocket book
• Ollama: an open supply instrument permitting to run domestically open-source giant language fashions, equivalent to Llama 2.
• NGrok: a instrument to reveal an area growth server to the Web with minimal effort.
• Enchanted: an open supply iOS/iPad cellular app for chatting with privately hosted fashions.

Google Colab

Google Colab, is a free cloud service hosted by Google that enables anybody to jot down and execute Python code via the browser.

Even with a free account, it supplies entry to a T4 GPU with 12GB RAM, which is essentially enough to run fashions like Mistral 7B or Llama 7B. Because it requires 8GB of RAM to run the 7B fashions.

Ollama