Hosting a Raspberry Pi Website Using Cloudflare Tunnels and LLMs

The deployment of Large Language Models (LLMs) on low-cost, single-board computers such as the Raspberry Pi is enabling a shift toward local AI hosting. By utilizing open-source software and secure networking tunnels, individuals and small-scale operators can run sophisticated models without relying on centralized cloud infrastructure.

This approach leverages the ARM processors found in Raspberry Pi hardware to execute models like Gemma, released by Google, and Llama 2, released by Facebook. The technical framework for this deployment typically involves the use of Ollama, an open-source project designed to simplify the creation and local execution of LLM models.

Technical Infrastructure and Deployment

Implementing a local LLM server requires a combination of containerization and configuration management tools. According to reporting by Madhan published on April 21, 2024, the setup involves using Docker-compose to install Ollama and Open WebUI as containers on the Raspberry Pi. Open WebUI serves as the user-friendly interface for interacting with the models.

The deployment process described by Madhan utilizes Ansible scripts to manage prerequisites and container setup. The workflow includes the following steps:

• Executing a Git checkout of the Open WebUI repository.
• Applying a Docker-compose template to the system.
• Running Docker Compose to initialize the containers.

Once the scripts are executed, the Open WebUI is accessible via the Raspberry Pi’s IP address on port 3000. Users can then log in as an administrator and navigate to the model settings page to download specific models, such as Gemma or Llama 2.

Secure Public Access via Cloudflare Tunnels

A primary challenge of self-hosting is exposing local services to the public internet securely. To address this, operators are using Cloudflare tunnels to connect locally running LLMs to the internet. This method is cited as a secure alternative to traditional networking configurations for exposing private network web services.

The use of Cloudflare tunnels extends beyond AI applications. Other implementations include the creation of pet cameras and the hosting of general web services on Raspberry Pi hardware, as noted in various technical guides published between February 20, 2024, and February 12, 2025.

The Economics of Self-Hosting

The move toward self-hosting represents a broader trend in the technology sector to reduce dependency on expensive cloud compute resources. By running open-source models on ARM-powered hardware, the cost of maintaining an AI instance is significantly lowered.

Users have reported that the process of establishing these tunnels and seeing a hosted website or service go live is heaps of fun. The integration of AI assistants, such as Claude, has been highlighted as a key factor in helping users navigate the complexities of self-hosting projects.

This synergy between open-source hardware, open-source models, and AI-assisted configuration is lowering the barrier to entry for deploying private, locally managed AI infrastructure.