Tag: GPU

Install and Run OLLAMA on Linux Machine

So many tech guys already share how to install the OLLAM. I wont say too details. Just a brief step for you.

  1. Prepare a Machine with good GPU, CPU and > 16G RAM. (Raspberry Pi can run with the Deepseek 1.5B, other………. Please chec my last post)
  2. Install update your linux repos.
    sudo apt-get update -y
    sudo apt-get upgrade -y
  3. Install Ollam by the follow command
    curl -fsSL https://ollama.com/install.sh | sh
  4. Run the LLM model, if you wont have the model at your machine, it will be download automatically.
    ollama run <model>
    e.g.: ollama run deepseek-r1:8b
  5. The model will be downloaded to /usr/share/ollama/.ollama/models/
  6. what model you can run? Check here
    https://ollama.com/search
  7. OLLAMA command line is a little bit similar with Docker, check this.

PS: You also can install OLLAMA at WINDOWS, please also check OLLAMA website.

Lets try your own AI locally!

#OLLAMA #Model #AI #CPU #GPU #CUDB #RAM #RaspberryPI #Docker

Deepseek 1.5b vs 8b version

Well, we all expect that 1.5b and 8b may have a different of AI’s knowledge.

We made a test,
1. 1.5b we are on Raspberry PI 4B 4G Ram.,
2. 8b on virtual machine with AMD Radeon and 16G Ram on Ubuntu.

We only ask a question.

“what is the difference between you and chatGPT”

  • 1. 1.5b versions

  • 2. 8b versions

The knowledge base of course 8b will be better. However, we will most concern of the resource usage. Can Raspberry PI CPU base can process this efficient?

#deepseek #AI #CPU #raspberrypi #GPU #nvidia #CUDB #AMD

Deepseek on Raspberry PI?????

Tech guys are interested in how AI and LLM model processing on an IOT, low power devices such as Raspberry PI.

But??!!!!

NO GPU!!!!!!!!!!!!

How to run the AI model????

OK, We dont want to talk about how to install and run on OLLAMA.

We have tried on 1.5b version of Deepseek on our PI4 4G RAM device.

Amazing that it works! However, you cannot expect the response time and token would be good enough for fast response.

By this kind of success, we can imagine that more other model can be running on CPU based IOT device. Therefore, will the home assistant widely adopt?

Let see……………….