Home Assistance Voice & Ollama Setup Guide - The Ultimate Local LLM Solution!

[00:00] a couple of weeks ago I showed off this

[00:02] the home assistant voice preview Edition

[00:05] an esp32 based smart speaker that if set

[00:08] up to use whisper and Piper handles

[00:11] voice commands and home automation fully

[00:15] locally through home assistant it's

[00:17] amazing but the one thing that it's

[00:19] missing is a proper conversation agent

[00:24] sure asking it to turn your lights on

[00:26] and set timers is useful but being able

[00:29] to ask questions and get answers would

[00:32] be really nice that is where AMA comes

[00:36] in ama is an open-source way to run

[00:39] large language models locally and it

[00:42] turns out it's ridiculously easy to set

[00:45] up and connect to home assistant voice

[00:48] so let me walk you through it and then

[00:50] demo just how useful it is first things

[00:53] first you'll need a Lama setup and

[00:55] running and you need a reasonably

[00:58] powerful PC for this ideally with a

[01:00] graphics card with lots of vram although

[01:03] it can run just on the CPU relatively

[01:07] quickly so long as you have enough

[01:09] system Ram too in my case I already have

[01:13] two home servers running already one for

[01:16] work and one for personal use so set up

[01:19] on my personal Nas which is running the

[01:21] latest stable build of true N scale and

[01:25] that latest stable build part is

[01:27] actually really important only in the

[01:30] last few months has tras finally

[01:33] migrated to using Docker containers as

[01:36] their Apps Manager that means installing

[01:39] a Lama is as easy as clicking install in

[01:42] the App Store setting up any parameters

[01:45] you might want like letting it use

[01:47] existing storage pools if you'd prefer

[01:50] and setting up how many CPU cores to

[01:52] allow it to use as well as how much RAM

[01:55] and if you can pass a GPU into it then

[01:59] that too then they're just hitting save

[02:02] that's pretty much it ready to go you

[02:05] can open the container shell and try and

[02:08] set up the various models and you know

[02:11] try them out right in the command line

[02:13] and this does actually give you a bit

[02:15] better control too although we'll come

[02:17] back to that in a second but you can

[02:19] also just leave it alone and head to

[02:21] home assistant you want to head to the

[02:23] devices and Integrations page and add

[02:26] the AMA integration put in the IP

[02:29] address of the Alama server and the

[02:31] correct port and then it will ask you

[02:34] what model you want to use here you can

[02:37] take your pick although this is where

[02:39] the command line interface might

[02:41] actually be beneficial if you don't have

[02:45] much RAM like my server did when I first

[02:47] set this up I've since doubled to 32 gig

[02:50] which still isn't that much for a ZFS

[02:52] server but either way you might find

[02:55] that you need to run the smaller

[02:57] versions of the models wherever possible

[03:01] these models all generally come with

[03:03] differing parameter counts the more

[03:05] parameters generally the better the

[03:08] responses but the harder they are to run

[03:11] and certainly the more memory it takes

[03:13] to run them too from home assistant

[03:17] there doesn't seem to be a way to

[03:19] install a particular parameter size

[03:22] version of a listed model it will just

[03:25] download the default which is often the

[03:28] largest one if if you need to pick a

[03:30] smaller one you might need to use the

[03:33] command line interface anyway which you

[03:35] can then run a Lama run model Name colon

[03:39] parameter size so as an example AMA Run

[03:43] Deep se- R1 colon

[03:45] 1.5b or if you just want to download the

[03:48] model but not run it swap run for pull

[03:52] once you've pulled the models you want

[03:53] or just pick the default in home

[03:55] assistant saving that will create an

[03:57] entity for the model in home assistant

[04:00] you can create as many of these entities

[04:02] as you want and try and swap them out

[04:05] see how the the responses work but once

[04:08] you've got that to be able to use the

[04:10] model with voice head to Voice Assistant

[04:12] click on local assistant and then change

[04:15] the conversation agent to the Alama

[04:17] model you want to use in the settings

[04:21] for that you can change the instruction

[04:23] given to the model along with the

[04:25] context window size Max message history

[04:28] and the keep alive typ personally I

[04:31] would recommend changing that from minus

[04:33] one which means permanently keep the

[04:35] model alive to something reasonable like

[04:38] a few minutes so it doesn't end up

[04:40] sprawling all over your systems Ram

[04:43] permanently there is also a setting to

[04:46] let the llm control home assistant

[04:49] although you will find that it's best to

[04:51] leave that set to No Control partially

[04:54] because a bunch of models don't support

[04:56] tools a required feature for the contr

[04:59] control to work and partially because of

[05:02] the setting in the main menu there

[05:05] prefer handling commands locally this

[05:08] means that commands like turn on lights

[05:11] are handled by home assistant or the

[05:13] home assistant agents while questions

[05:16] that it doesn't know the answer to are

[05:18] passed off to the llm that means those

[05:21] commands are run faster and more

[05:23] efficiently and it keeps the

[05:25] compatibility problem at Bay 2 so now

[05:28] with that set up we've successfully

[05:30] connected home assistant voice to a

[05:32] locally run large language model it's

[05:35] remarkably simple and works pretty well

[05:39] okay

[05:40] Nabu how long is an inch in

[05:54] cenm 1 in is approximately equal to 2.54

[05:58] cm

[05:59] there you go now responses do take a bit

[06:02] longer than the built-in conversation

[06:04] agent because it's now doing speech to

[06:06] text generating response from the llm

[06:09] and then text to speech to voice the

[06:12] answer but considering this is all

[06:14] running locally I'm pretty happy with

[06:17] that the one thing that you'll want to

[06:19] try out and consider is which model to

[06:22] use well deep seek R1 is the new hotness

[06:26] the r in the name may as well stand for

[06:28] reasoning because the responses it gives

[06:31] contain an awful lot of text for a

[06:34] relatively simple question asking it

[06:37] what you know an inch in centimeters is

[06:40] spits out two large paragraphs of

[06:43] thinking followed by a single sentence

[06:46] answer if you could maybe filter out

[06:48] everything in the think tags that might

[06:51] work but for me I just opted to use Lama

[06:54] 3.2 instead that seems to work pretty

[06:57] well the only other thing that I'd like

[06:59] to add to this is the ability to search

[07:01] the web although that doesn't seem like

[07:04] the most simple of additions so I'll

[07:07] have to put that one on hold for the

[07:09] time being in short then setting up a

[07:12] llama and home assistant voice is

[07:14] remarkably easy and the results well a

[07:17] little on the slow side are great let me

[07:20] know in the comments if you set this up

[07:22] and how you get on with it and which

[07:25] model you end up choosing as well so

[07:27] yeah that's how to set it up if you want

[07:29] to see more videos like this one you can

[07:30] hit the Subscribe button check out

[07:31] plenty of other videos in the end cards

[07:33] including the home assistant voice

[07:35] preview Edition review that'll be in the

[07:37] cards on the end cards and in the cards

[07:39] above and otherwise that's pretty much

[07:41] it hope you enjoyed the video thank you

[07:43] for watching if you want to check out my

[07:44] own Open Source Hardware the open source

[07:46] response time tool and open source

[07:48] latency testing tool those are available

[07:50] at os.com Linked In the description

[07:53] nowise yeah thanks for watching hope you

[07:54] enjoyed it we'll see you on the next

[07:56] video