DEV Community

#localllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64
xbill profile

Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

#gemma #llamacpp #cuda #localllm
4
Comments
13 min read
Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s
jamilxt profile

Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s

#ai #localllm #performance #llm
1
Comments
7 min read
Orquesta CLI: Local LLM Management with Dashboard Sync
orquesta_live profile

Orquesta CLI: Local LLM Management with Dashboard Sync

#orquestacli #localllm #dashboardsync #ollama
Comments
6 min read
Estimating tokens/s for Mixture-of-Experts models: active parameters, plus a routing term
alexfank profile

Estimating tokens/s for Mixture-of-Experts models: active parameters, plus a routing term

#llm #localllm #machinelearning #performance
1
Comments
5 min read
EmbeddingGemma 2 puts multimodal search on a phone
techaiwire profile

EmbeddingGemma 2 puts multimodal search on a phone

#google #openweights #localllm #huggingface
5
Comments
4 min read
Qwen3.8-27B on one RTX 3090, and the setting its README says chat clients should leave at default
renolu profile

Qwen3.8-27B on one RTX 3090, and the setting its README says chat clients should leave at default

#vllm #localllm #speculativedecoding #quantization
Comments
4 min read
Still carrying the diagnosis
thekilteddev profile

Still carrying the diagnosis

#localllm #debugging #buildinpublic #opensource
Comments
8 min read
Claude CLI 401 Unauthorized Refresh Token Issue
devlog profile

Claude CLI 401 Unauthorized Refresh Token Issue

#clitools #localllm #troubleshooting
Comments
3 min read
Discord bot not responding but still running: how to catch it
devlog profile

Discord bot not responding but still running: how to catch it

#automation #localllm #macos
Comments
3 min read
Claude Code MCP setup and usage rules
devlog profile

Claude Code MCP setup and usage rules

#clitools #macos #localllm
Comments
3 min read
Clone your own voice with a 17-second recording
devlog profile

Clone your own voice with a 17-second recording

#contentops #localllm #macos
Comments
2 min read
Prompt engineering: How to fix instructions that AI ignores
devlog profile

Prompt engineering: How to fix instructions that AI ignores

#promptengineering #ai #localllm
Comments
3 min read
Local Coding Agent: Weak Model? No, Bad Plumbing
eryk_kubiak_c9663554a1ff4 profile

Local Coding Agent: Weak Model? No, Bad Plumbing

#llm #ai #localllm #agents
Comments 1
13 min read
What Happens When You Ask an LLM a Question
vishnuhd profile

What Happens When You Ask an LLM a Question

#ai #localllm #llmbasics #gpu
Comments
8 min read
Run vLLM on Kubernetes with Minikube, WSL2 and NVIDIA GPU
vishnuhd profile

Run vLLM on Kubernetes with Minikube, WSL2 and NVIDIA GPU

#ai #localllm #vllm #kubernetes
Comments
13 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.