The objective of this article is to run different large language models tools to analyze local docs.
LM Studio is a desktop app for developing and experimenting with LLMs on your computer.
Key functionality
- A desktop application for running local LLMs
- A familiar chat interface
- Search & download functionality (via Hugging Face 🤗)
- A local server that can listen on OpenAI-like endpoints
- Systems for managing local models and configurations
It is an all-in-one zero-setup private application for local LLMs, RAG and AI Agents with embedding models and vector database supportability
An open-source ecosystem used for integrating LLMs into applications without paying for a platform or hardware subscription. It was created by Nomic AI, an information cartography company that aims to improve access to AI resources.
Designed to run on modern to relatively modern PCs without needing an internet connection or even a GPU! This is possible since most of the models provided by GPT4All have been quantized to be as small as a few gigabytes, requiring only 4–16GB RAM to run.
git clone https://github.com/AshleyDhevalall/llm-tools-demo.git
- Download LM Studio
Tip
-
Open LM Studio. Please patient while the application opens...
-
Download model (Click
My Modelson left menu). Ensure you have selected 'Developer' as the role You can search for models by keyword (e.g. llama, gemma, lmstudio), or by providing a specific user/model string.
You can even insert full Hugging Face URLs into the search bar!. Please patient while to model downloads...
- Click
Chaton left menu
Select model
-
Navigate to the folder in Clone Repository and select the
samples\UsingSQLWithPowerShellExamplesv1.ps1file -
Enter your prompt and click
Send
what passwords are used?
Sample chat response
Important
Always verify the accuracy of the results
Docs
Download Models
Chat
Chat with Documents
- Download AnythingLLM
Tip
-
Open AnythingLLM. Please patient while the application opens...
-
Create new workspace. Enter name for collection and click
Save
Tip
- Upload files. Navigate to the folder in Clone Repository and select the
samples\romeo-and-juliet.pdffile
Warning
PDF, TXT, DOCX, Word docs and more. Unfortunately AnythingLLM does not support .ps1 file :(
Drag and drop your files into the box shown below
Once complete, select the file and click Move to Workspace
Click Save and Embed
Please patient while the files complete embedding...
Close the My Documents window once the embedding is complete
- Enter your prompt and click
Send prompt message to workspaceicon
Who are the characters in Romeo and Juliet?
Sample prompt response
Important
Always verify the accuracy of the results
Could not respond to message. fetch failed
Setup
Configuration
System Requirements
FAQ
- Download GPT4All Desktop
Tip
-
Open GTP4All. Please patient while the application opens...
-
Adding new model (Click
Modelson left menu) and then click+ Add Modelbutton
Tip
- Search for
Llama 3.2 1B Instructand then clickDownload. Please patient while to model downloads...
- Configure settings
Tip
6.1 Click LocalDocs on left menu. Provide a Collection name eg. Sample Test
Tip
6.2 Click Browse.
Navigate to the folder in Clone Repository and select the samples folder. Please patient while the embeddings complete...
-
Click
New chaton left menu. Ensure you select the files from theLocal Docspanel on the right sideEnter your prompt and click
Send
what passwords are used?
Tip
Sample chat response
Important
Always verify the accuracy of the results
GTP4All has an option that allows you to view the source of the files in the Chat response
Uploading large files will result in an increased duration for the embedding to complete. Alternatively try a smaller subset of docs
Ensure that you have included the correct file extension
This usually happens when using models that required more RAM than is available on your system. Try using a smaller model
Quickstart
Wiki
FAQ
Settings
What Is GPT4All and How Does It Work?
GTP4All
AnythingLLM
Joshua Stenhouse
Jayric Maning
anakin.ai
LM Studio