host ALL your AI locally

About this video

NetworkChuck builds a fully local, private AI setup that runs entirely on your own hardware with a ChatGPT-style web interface — no cloud AI account required. The stack has three layers. First, Ollama is the engine that runs the models: installed with one command (on Windows he insists on the Linux version via WSL rather than the Windows preview), it serves an API on localhost port 11434, and models are added with commands like ‘ollama pull llama2’ — he also pulls Code Gemma and the multimodal LLaVA model, and notes Ollama can use both of his GPUs at once. Second, Open WebUI runs in a Docker container (port 8080) and provides the polished chat front end: chat history, switching or even combining models mid-conversation, file uploads for summarization, image input for multimodal models, and an admin panel where the first account created becomes admin, new users sit pending until approved, and admins can whitelist which models each user may touch. Third, Stable Diffusion via the AUTOMATIC1111 web UI (installed with pyenv-managed Python 3.10, served on port 7860, relaunched with –listen and –api flags) plugs into Open WebUI’s image settings so images generate inline in chat. A major motivation is control for his kids: using Ollama Modelfiles, he builds a custom model for his daughter Chloe (she named it Deborah) whose system prompt makes it act as a tutor that guides homework instead of writing essays, and he shows it refusing to write a paper. He also connects the local models into his Obsidian notes via the BMO Chatbot community plugin, including referencing the current note. The demo runs on a laptop, but he shows off ‘Terry,’ his dedicated AI server — per the creator: a Ryzen 9 7950X, 128 GB of DDR5-6000 RAM, two liquid-cooled RTX 4090s with 24 GB each, and a 1600 W power supply, running Pop!_OS after Ubuntu refused to install. Sponsor IT Pro (ACI Learning) is pitched in-video with a creator-claimed 30% discount code.

Key takeaways

• The core recipe is Ollama (model runner, API on port 11434) plus Open WebUI in Docker for a polished, private, ChatGPT-like interface on hardware you already own.
• Models are interchangeable parts: ‘ollama pull’ fetches Llama 2, Code Gemma, LLaVA for image input, and others, and Open WebUI can switch or combine models inside one conversation.
• Open WebUI’s admin controls matter for families: first signup becomes admin, later users need approval, and admins can whitelist models per user and block chat deletion.
• Custom Modelfiles with a restrictive system prompt turn a general model into a guarded tutor — his daughter’s ‘Deborah’ model refuses to write essays and instead guides the work.
• Stable Diffusion (AUTOMATIC1111, port 7860, needs the –listen and –api flags) integrates into Open WebUI for inline image generation, and a BMO Chatbot plugin brings the same local models into Obsidian notes.

Episode details: published May 3, 2024 · 24:20 long. Summary written by Self Growth Videos from the episode itself. Business results and any income figures mentioned are NetworkChuck’s own claims, shared for education — they are not typical or guaranteed.

← Previous: White House says NO DEGREE!! · Next: monitor all your stuff RIGHT NOW!! →

All NetworkChuck videos · NetworkChuck profile · NetworkChuck on YouTube