Tech guys are interested in how AI and LLM model processing on an IOT, low power devices such as Raspberry PI.
But??!!!!
NO GPU!!!!!!!!!!!!
How to run the AI model????
OK, We dont want to talk about how to install and run on OLLAMA.
We have tried on 1.5b version of Deepseek on our PI4 4G RAM device.
Amazing that it works! However, you cannot expect the response time and token would be good enough for fast response.
By this kind of success, we can imagine that more other model can be running on CPU based IOT device. Therefore, will the home assistant widely adopt?
Let see……………….