Using ngrok with Ollama
Ollama is a locally deployed AI model runner, designed to allow users to download and execute large language models (LLMs) locally on your machine. A perfect pairing for ngrok. By combining Ollama with ngrok, you can give your local Ollama an endpoint on the internet, enabling remote access and integration with other applications.
Prerequisites
1. Install ngrok
Download and install the ngrok CLI from the official website for your OS.
For macOS using Homebrew:
Loading…
3. Install and run Ollama on your machine
Install Ollama by following the instructions on the Ollama website.
Once installed, start the Ollama server:
Loading…
By default, Ollama will start on
Loading…
4. Create an endpoint for your Ollama server using ngrok to create an endpoint
In a new terminal window, start an ngrok tunnel to your local Ollama port:
Loading…
ngrok will generate a public forwarding URL like:
Loading…
This URL now provides internet access to your local Ollama instance.
5. Use your Ollama instance from anywhere
You can now send requests to your Ollama server from anywhere using the ngrok URL. For example:
Loading…
Be sure to replace
Loading…
with your actual ngrok URL.
Last thing, you now have a public endpoint for your Ollama instance, which means anyone on the internet could find it and use it.
6. Protect your Ollama instance with basic auth
You may not want everyone to be able to access your LLM. ngrok can quickly add authentication to your LLM without any changes. Explore Traffic Policy to understand all the ways ngrok can protect your endpoint.
The following example demonstrates how to use the basic-auth Traffic Policy action to only allow visitors with the credentials user:password1 or admin:password2 to access your app.
Loading…
You can test your traffic policy by sending the following request to your app.
Loading…
This example uses dXNlcjpwYXNzd29yZDE=, which is a base64 encoded version of user:password1.
You should get the following response:
Loading…
If you send the same request without the Authorization header, you should receive a 401 Unauthorized response.
Done!
Your personal LLM is now locked down to only accept authenticated users.