Can something like this be done on a NAS and somehow maintained there? I don't understand all of the instructions yet, will have to do a lot of homework first, sorry if this question is relatively stupid.
This is excellent, but is there a version (or path to muddle through...) for doing this in Linux, with Python 3.13? What are the blockers/problems the articles mentions?
If you go the github page I linked, its own README.md which automatically opens below the file structure is written with Linux in mind. Simply follow it. And since virtualenv in fact does work in the expected way on Linux, you can simply start from the newest Python, and work your way down the versions until you find one that works. It might even be 3.13 . But remember: The way virtualenv works is that the entire Python interpreter is installed in the project directory. Completely independent of what your main Python of the operating system is. So, it doesn't ultimately matter much which version you have to settle on. Your main Python will always remain 3.13. EDIT: You'll know that a particular version did not work from the fact that trying to install requirements.txt with pip gives you an error when trying to build a particular dependency. Then delete the venv directory, and build a new one with one python version down, for example 3.13 to 3.12, until you find one that is able to install the entire requirements.txt . It's not Deepseek's fault that a new Python version is not supported, but the fault of the maintaners of the dependency that failed. It'll get solved eventually. This is unfortunately typical of Python projects. Dependencies are maintained by different groups of people, and some are much more active than others. If even one project doesn't get updated when a Python version breaks it, then projects that rely on it will fail. Then you just do the same process: Go down the versions in your venv until one works. This is why virtualenv exists. This would be completely intolerable if you had to change your operating system Python because of one problem project. Virtualenv isolates the problem.
I had to add a few more installs before I could get "streamlit run app.py" to work. Each time I received an error I just installed the package in the error message:
pip install streamlit
pip install networkx
pip install langchain_core
pip install langchain_community
pip install langchain_ollama
pip install rank_bm25
pip install sentence_transformers
pip install hf_xet
pip install faiss-cpu
It's working now, and I imported the text from part 2. The only issue now is that the responses appear hidden and I have to highlight it or inspect the source to see the responses.
Initial results do not look promising:
Tell me about Napoleon Bonaparte
Napoleon Bonaparte, born Élisée de Grayson, was a pivotal figure in European history as both a military leader and political statesman. His early life was marked by financial hardship and a passion for revolution. He rose through the French military, achieving significant successes like the French and Indian War (1756–1763) and the American Revolutionary War.
Military Successes: Napoleon's military prowess was undeniable. His victories over Russia and Austria were pivotal moments that bolstered France's reputation and economic prosperity. These achievements not only elevated the morale of his troops but also showcased Napoleon's tactical brilliance, solidifying his place as a formidable military leader.
As far as your results go, you could try your luck with doubling the parameters, which are sort of virtual neurons. Find the ollama pull line and replace 7b with 14b. Then find a file called .env in the project directory, and change 7b to 14b. Then just run again. It will probably take more than twice the amount of time. If it's too slow, then do "ollama rm" instead of pull on the 14b model, and edit the .env back.
All of them should have been installed by this line: "python3.11.exe -m pip install -r .\requirements.txt" . Perhaps you executed it in the wrong directory, or you just thought you had executed it but never actually did. EDIT: OR, you used the specific Python version in the call only in some of those commands. So, the requirements got installed into a different version than what is being called by Streamlit.
for programmer newbie but v. comfortable with *nix command line, you're saying the linked README is soup to nuts for a Linux HW & SW setup? (will go there and check...)
Also - for GPU stuff -- any thoughts on sourcing used/lower-bin/older generation GPUs online stores?
If you're not making it a server for customers, but just for your own use or otherwise just one user at a time, you can find a RTX 3060 for about 200 bucks, and it will suffice. Used easily, but since that's pretty much stock that nobody wants anymore, you're pretty much guaranteed to find even a brand new one for the same price somewhere.
"And if you’re interested in providing material for a guest post that demonstrates how you created an AI work or reviews an AI technology, please subscribe and leave a message here in the comments."
In the comments. Plural. Here is fine. Just send an email, you have it.
And the comments have not been turned off in the original post. It's a quirk of the Substack setup, but for some reason, comments do not work on the first "coming soon" autopost. I should have created a new post rather than repurposing the original post.
Hi, it's the author, in all other regards you just follow the project Github page's own README just like Linux, but the problem part is getting GPU acceleration. Older Macs have Nvidia, in which case you'd just install CUDA and CuDNN like Windows. Newer ones can have either AMD or Metal. Now, I don't have a Mac so I can't test the instructions, but even if I did, it would just work for one brand of GPU. So, if you can get the component called torch running on GPU with the info on this page, everything else will work exactly like Linux. You will get it working on CPU mode, but it will probably be horribly slow. GPU architecture is perfect for the kind of calculations neural networks require. CPU is the exact opposite. https://developer.apple.com/metal/pytorch/
Can something like this be done on a NAS and somehow maintained there? I don't understand all of the instructions yet, will have to do a lot of homework first, sorry if this question is relatively stupid.
This is excellent, but is there a version (or path to muddle through...) for doing this in Linux, with Python 3.13? What are the blockers/problems the articles mentions?
If you go the github page I linked, its own README.md which automatically opens below the file structure is written with Linux in mind. Simply follow it. And since virtualenv in fact does work in the expected way on Linux, you can simply start from the newest Python, and work your way down the versions until you find one that works. It might even be 3.13 . But remember: The way virtualenv works is that the entire Python interpreter is installed in the project directory. Completely independent of what your main Python of the operating system is. So, it doesn't ultimately matter much which version you have to settle on. Your main Python will always remain 3.13. EDIT: You'll know that a particular version did not work from the fact that trying to install requirements.txt with pip gives you an error when trying to build a particular dependency. Then delete the venv directory, and build a new one with one python version down, for example 3.13 to 3.12, until you find one that is able to install the entire requirements.txt . It's not Deepseek's fault that a new Python version is not supported, but the fault of the maintaners of the dependency that failed. It'll get solved eventually. This is unfortunately typical of Python projects. Dependencies are maintained by different groups of people, and some are much more active than others. If even one project doesn't get updated when a Python version breaks it, then projects that rely on it will fail. Then you just do the same process: Go down the versions in your venv until one works. This is why virtualenv exists. This would be completely intolerable if you had to change your operating system Python because of one problem project. Virtualenv isolates the problem.
helpful, currently a smidge over my non SDE head, and gives guidance for how to grow/proceed. thanks
I had to add a few more installs before I could get "streamlit run app.py" to work. Each time I received an error I just installed the package in the error message:
pip install streamlit
pip install networkx
pip install langchain_core
pip install langchain_community
pip install langchain_ollama
pip install rank_bm25
pip install sentence_transformers
pip install hf_xet
pip install faiss-cpu
It's working now, and I imported the text from part 2. The only issue now is that the responses appear hidden and I have to highlight it or inspect the source to see the responses.
Initial results do not look promising:
Tell me about Napoleon Bonaparte
Napoleon Bonaparte, born Élisée de Grayson, was a pivotal figure in European history as both a military leader and political statesman. His early life was marked by financial hardship and a passion for revolution. He rose through the French military, achieving significant successes like the French and Indian War (1756–1763) and the American Revolutionary War.
Military Successes: Napoleon's military prowess was undeniable. His victories over Russia and Austria were pivotal moments that bolstered France's reputation and economic prosperity. These achievements not only elevated the morale of his troops but also showcased Napoleon's tactical brilliance, solidifying his place as a formidable military leader.
As far as your results go, you could try your luck with doubling the parameters, which are sort of virtual neurons. Find the ollama pull line and replace 7b with 14b. Then find a file called .env in the project directory, and change 7b to 14b. Then just run again. It will probably take more than twice the amount of time. If it's too slow, then do "ollama rm" instead of pull on the 14b model, and edit the .env back.
All of them should have been installed by this line: "python3.11.exe -m pip install -r .\requirements.txt" . Perhaps you executed it in the wrong directory, or you just thought you had executed it but never actually did. EDIT: OR, you used the specific Python version in the call only in some of those commands. So, the requirements got installed into a different version than what is being called by Streamlit.
for programmer newbie but v. comfortable with *nix command line, you're saying the linked README is soup to nuts for a Linux HW & SW setup? (will go there and check...)
Also - for GPU stuff -- any thoughts on sourcing used/lower-bin/older generation GPUs online stores?
thanks.
If you're not making it a server for customers, but just for your own use or otherwise just one user at a time, you can find a RTX 3060 for about 200 bucks, and it will suffice. Used easily, but since that's pretty much stock that nobody wants anymore, you're pretty much guaranteed to find even a brand new one for the same price somewhere.
Anticipation of this article was the exact reason I signed up, and this Substack did not disappoint. Thank you!
Wow, this is has BIG potential. Thanks for the post!
Thank you, MK!
@voxday:
"And if you’re interested in providing material for a guest post that demonstrates how you created an AI work or reviews an AI technology, please subscribe and leave a message here in the comments."
https://aicentral.substack.com/p/coming-soon
The comments have been turned off on the original post. Where should we leave a message if interested in contributing?
In the comments. Plural. Here is fine. Just send an email, you have it.
And the comments have not been turned off in the original post. It's a quirk of the Substack setup, but for some reason, comments do not work on the first "coming soon" autopost. I should have created a new post rather than repurposing the original post.
Thank you for this. Would greatly appreciate a Mac version as well. Or would it be best to just network in a gaming PC?
Hi, it's the author, in all other regards you just follow the project Github page's own README just like Linux, but the problem part is getting GPU acceleration. Older Macs have Nvidia, in which case you'd just install CUDA and CuDNN like Windows. Newer ones can have either AMD or Metal. Now, I don't have a Mac so I can't test the instructions, but even if I did, it would just work for one brand of GPU. So, if you can get the component called torch running on GPU with the info on this page, everything else will work exactly like Linux. You will get it working on CPU mode, but it will probably be horribly slow. GPU architecture is perfect for the kind of calculations neural networks require. CPU is the exact opposite. https://developer.apple.com/metal/pytorch/
Thanks for sharing your knowledge.
You can try the Linux instructions on the Mac terminal