Building a Local Deepseek I

Instructions on installing your own local neural network

MK is an expert Linux programmer based in Finland. After building his own private Deepseek on Linux, he graciously decided to provide Windows-based AI Central readers how it can also be done on Windows. In Part II, he demonstrated how to set up your own data sources for your own local neural network.

Until recently, training an AI on a set of research data has been so resource-intensive that it has been entirely out of reach for home users even for smaller ("distilled") models intended for ordinary gaming computers. However, these days there is a methodology called Retrieval-Augmented Generation, RAG for short, that can achieve something very close to the effect of training in a relatively tiny portion of the time. The trade-off is that the understanding of the data is not as deep, and the data has to be processed every time the AI is launched.

With an average gaming PC with an NVIDIA GPU that's in the RTX 3000 -series or newer, you can expect it to spend about 10 minutes, assuming you use a modest 7 billion parameter model. Parameters can be thought of as virtual brain cells. With a better computer, 14 billion is also realistic, especially if you are asking just a few but important questions. Since you are having the AI focus on a set of data that is extremely limited compared to cloud-based AI's, the normally expected half trillion parameter counts aren't important. You need them only when the AI has to essentially know the entire contents of the internet. For one set of books, 7 to 14 billion is sufficient. If you choose to get serious about using a locally installed AI, then you'd install it on a dedicated Linux server and keep it constantly running, which mitigates the problem to essentially zero.

This article is aimed at Windows users who have never installed a local neural network, and have little to no experience of Python. Linux users will be able to handle the project's own README. You'll need a relatively recent NVIDIA GPU. (At the time of this writing, AMD is seriously behind when it comes to easily accessible development environments.) The instructions are written with the assumption that the user is capable of installing the CUDA Toolkit 12.9.1.

I'll also skip the part about virtualenv. It's awkward to use on Windows, as I found out. The only scenario where virtualenv benefits you is if you are already using the specific Python version 3.11 for other projects, and you want to keep the libraries separate. That probably means you're a pro user, and should also opt for the project README. This is meant as an easy introduction to running your first neural network with zero experience. I'm choosing Python 3.11 because it's the newest version that doesn't give you any problems with the project dependencies. A few reboots may be needed while you install all the different programs below. The installer should inform you of this.

Make sure your ordinary display drivers are in the latest version. Then, we'll need some development drivers for the GPU that we don't normally need for games. Install version 12.9.1 here:

https://developer.nvidia.com/cuda-toolkit-archive .

Install an additional set of NVIDIA neural network drivers here:

https://developer.nvidia.com/cudnn

Install Python 3.11: https://apps.microsoft.com/detail/9nrwmjp3717k?hl=en-US&gl=US using View in Store or the installer file. Open Windows PowerShell by typing powershell in the Start menu search, and make sure that when you write python3.11 and press TAB, it completes to the corresponding .exe . On Windows, you get out of Python by pressing Ctrl-z and then enter. Again, at the time of writing, any newer Python will give you problems if you follow the instructions as-is.

Install the command line program Git from this link:

https://git-scm.com/downloads/win

Install Ollama, which is a manager for downloading and running a whole bunch of different neural network architectures on the same computer. You may want to disable the automatic startup when Windows launches, though, by searching the Start menu with "startup" and then from Startup Apps system setting, turning the switch next to Ollama off.

Download it here: https://ollama.com and then in in Powershell, execute the following:

ollama pull deepseek-r1:7b

ollama pull nomic-embed-text

Create a directory such as C:\gitprojects and navigate to it in Powershell. You'll download this project as its subdirectory using Git. Execute:

cd c:\gitprojects

Here is the actual AI project, if you want to see its own README file: https://github.com/SaiAkhil066/DeepSeek-RAG-Chatbot

To download it, execute:

git clone https://github.com/SaiAkhil066/DeepSeek-RAG-Chatbot.git

cd DeepSeek-RAG-Chatbot

First you need to upgrade your Python package manager. Installers always deliver obsolete versions.

python3.11.exe -m pip install --upgrade pip

Next, we'll install everything this project needs. You need to be in the project directory. Now, there is an awkward part on Windows: you need to pick up the directory from the warnings where it installs all this, and add it to your PATH environment variable. Paint it with your mouse in the PowerShell window and press Ctrl+c. It's the same path in all warnings. So, first execute this and then find those lines and press Ctrl-c on them:

python3.11.exe -m pip install -r .\requirements.txt

Now the problem path should be on your clipboard. In Start menu search, type "environ" and choose Edit the system environment variables. The bottom button in the window is Environment Variables. Click it. The top area contains paths that are for your Windows user only, not the entire system. We'll place it there. One of the Variables on the left column is Path. Double-click the Value-field on the right.

Now there is a New-button, allowing you to paste the directory with Ctrl-V. It will look something akin to (but not exactly like) this: C:\Users\[username]\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts

For PowerShell to read your addition, it needs to be closed and started again from Start menu. Then, to get back to the project directory, execute:

cd c:\gitprojects\DeepSeek-RAG-Chatbot\

To make sure Ollama is active, execute the following. If it gives an error about only one usage of a socket address, it was in fact running and there is no problem. Otherwise it starts now.

ollama serve

Everything you've done above is just standard setup that you'd be doing to your operating system, whatever you're going to do with neural networks. So, this is probably the easiest way to see what an actual AI project contains. Everything that is specific to how we're using DeepSeek here is in the project directory. We were only downloading perfectly standard components until that.

So, finally, the project itself is started by executing this. It must always be in the project directory. Starting ollama and executing this command are the only steps you will do after you've gone through all of the above once. Note that neural networks are incredible resource hogs, and it might take minutes until anything happens. Also, on the first run, Windows will ask permission for it to use the network. Don't worry, it's for local use only. It allows the use of your ordinary browser as the GUI to the project. When you're done asking all your questions, in order to free the system memory, close the PowerShell window from the close button in the window corner. But as for now, execute:

streamlit run app.py

If all went well, your browser automatically opened the user interface.

---

In Part II, MK explains how to set up your own data sources which your new neural network will utilize. We would very much welcome any corrections or additional insights discovered by those who attempt to utilize these instructions and run into problems or other infelicities.

NOTE: Macintosh installations are complicated by the fact that Apple uses three difrerent GPU systems, Metal, AMD, and NVIDIA. It is necessary to make sure the correct GPU is configured for this setup to function correctly.