craft fix
This commit is contained in:
305
nodejs/OLLAMA_SETUP.md
Normal file
305
nodejs/OLLAMA_SETUP.md
Normal file
@@ -0,0 +1,305 @@
|
||||
# Ollama Integration Guide
|
||||
|
||||
This project now supports Ollama as an AI backend alongside Google Gemini, with **per-bot configuration** allowing you to mix providers and personalities across multiple bots.
|
||||
|
||||
## Configuration Hierarchy
|
||||
|
||||
AI settings are merged in this order:
|
||||
1. **Global defaults** in `conf/base.js` → `ai` object
|
||||
2. **Bot-specific overrides** in `conf/secrets.js` → `mc.bots.{botName}.plugins.Ai`
|
||||
|
||||
## Configuration
|
||||
|
||||
### Global Defaults (Optional)
|
||||
|
||||
Edit `conf/base.js` to set defaults for all bots:
|
||||
|
||||
```javascript
|
||||
"ai":{
|
||||
// Default provider (can be overridden per-bot)
|
||||
"provider": "gemini", // or "ollama"
|
||||
|
||||
// Gemini API key (used by Gemini provider)
|
||||
"key": "<configure in conf/secrets.js>",
|
||||
|
||||
// Ollama settings (used by Ollama provider)
|
||||
"baseUrl": "http://localhost:11434",
|
||||
"model": "llama3.2",
|
||||
"timeout": 30000,
|
||||
|
||||
// Generation settings (applies to both providers)
|
||||
"temperature": 1,
|
||||
"topP": 0.95,
|
||||
"topK": 64,
|
||||
"maxOutputTokens": 8192,
|
||||
"interval": 20,
|
||||
// ... prompts
|
||||
}
|
||||
```
|
||||
|
||||
### Per-Bot Configuration
|
||||
|
||||
Edit `conf/secrets.js` to configure each bot individually:
|
||||
|
||||
#### Example 1: Bot using default global settings
|
||||
|
||||
```javascript
|
||||
"art": {
|
||||
"username": "art@vm42.us",
|
||||
"commands": ['fun', 'invite', 'default'],
|
||||
"auth": "microsoft",
|
||||
"plugins": {
|
||||
"Ai":{
|
||||
"promptName": "helpful",
|
||||
// Uses global provider settings from base.js
|
||||
}
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
#### Example 2: Bot using specific Ollama instance
|
||||
|
||||
```javascript
|
||||
"ayay": {
|
||||
"username": "limtisengyes@gmail.com",
|
||||
"commands": ['fun', 'invite', 'default'],
|
||||
"auth": "microsoft",
|
||||
"plugins": {
|
||||
"Ai":{
|
||||
"promptName": "asshole",
|
||||
"provider": "ollama",
|
||||
"baseUrl": "http://192.168.1.50:11434", // Remote Ollama
|
||||
"model": "llama3.2:7b",
|
||||
"interval": 25,
|
||||
}
|
||||
}
|
||||
},
|
||||
```
|
||||
|
||||
#### Example 3: Bot using Gemini with custom settings
|
||||
|
||||
```javascript
|
||||
"nova": {
|
||||
"username": "your@email.com",
|
||||
"auth": "microsoft",
|
||||
"commands": ['default', 'fun'],
|
||||
"plugins": {
|
||||
"Ai":{
|
||||
"promptName": "helpful",
|
||||
"provider": "gemini",
|
||||
"model": "gemini-2.0-flash-exp",
|
||||
"temperature": 0.7,
|
||||
}
|
||||
}
|
||||
},
|
||||
```
|
||||
|
||||
### Multiple Bots with Different Providers
|
||||
|
||||
You can run multiple bots with different providers simultaneously:
|
||||
|
||||
```javascript
|
||||
// conf/secrets.js
|
||||
"bots": {
|
||||
"bot1": {
|
||||
"plugins": {
|
||||
"Ai": {
|
||||
"promptName": "helpful",
|
||||
"provider": "gemini", // Uses Google Gemini
|
||||
"model": "gemini-2.0-flash-exp",
|
||||
}
|
||||
}
|
||||
},
|
||||
"bot2": {
|
||||
"plugins": {
|
||||
"Ai": {
|
||||
"promptName": "asshole",
|
||||
"provider": "ollama", // Uses local Ollama
|
||||
"baseUrl": "http://localhost:11434",
|
||||
"model": "llama3.2",
|
||||
}
|
||||
}
|
||||
},
|
||||
"bot3": {
|
||||
"plugins": {
|
||||
"Ai": {
|
||||
"promptName": "Ashley",
|
||||
"provider": "ollama", // Uses remote Ollama
|
||||
"baseUrl": "http://192.168.1.50:11434",
|
||||
"model": "mistral",
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Mixing Personalities and Models
|
||||
|
||||
Each bot can have:
|
||||
- **Different provider** (Gemini or different Ollama instances)
|
||||
- **Different model** (llama3.2, mistral, qwen2.5, etc.)
|
||||
- **Different personality** (helpful, asshole, Ashley, custom)
|
||||
- **Different settings** (temperature, interval, etc.)
|
||||
|
||||
```javascript
|
||||
"helpfulBot": {
|
||||
"plugins": {
|
||||
"Ai": {
|
||||
"promptName": "helpful",
|
||||
"provider": "ollama",
|
||||
"baseUrl": "http://server1:11434",
|
||||
"model": "llama3.2:3b",
|
||||
"temperature": 0.5,
|
||||
"interval": 15,
|
||||
}
|
||||
}
|
||||
},
|
||||
"toxicBot": {
|
||||
"plugins": {
|
||||
"Ai": {
|
||||
"promptName": "asshole",
|
||||
"provider": "ollama",
|
||||
"baseUrl": "http://server2:11434",
|
||||
"model": "llama3.2:70b",
|
||||
"temperature": 1.2,
|
||||
"interval": 30,
|
||||
}
|
||||
}
|
||||
},
|
||||
```
|
||||
|
||||
## Ollama Setup
|
||||
|
||||
### Install Ollama
|
||||
|
||||
```bash
|
||||
# Linux/macOS
|
||||
curl -fsSL https://ollama.com/install.sh | sh
|
||||
|
||||
# Or download from https://ollama.com/download
|
||||
```
|
||||
|
||||
### Pull Models
|
||||
|
||||
```bash
|
||||
# Recommended for chat bots:
|
||||
ollama pull llama3.2
|
||||
ollama pull mistral
|
||||
ollama pull qwen2.5
|
||||
|
||||
# Specific sizes for performance tuning:
|
||||
ollama pull llama3.2:3b # Fast, lightweight
|
||||
ollama pull llama3.2:7b # Good balance
|
||||
ollama pull llama3.2:70b # Smarter, slower
|
||||
```
|
||||
|
||||
### Start Ollama Server
|
||||
|
||||
```bash
|
||||
# Local (only)
|
||||
ollama serve
|
||||
|
||||
# Allow remote connections (for multiple servers)
|
||||
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
||||
```
|
||||
|
||||
### Configure Remote Ollama
|
||||
|
||||
To use Ollama on another machine:
|
||||
|
||||
1. On the Ollama server:
|
||||
```bash
|
||||
OLLAMA_HOST=0.0.0.0:11434 ollama serve
|
||||
```
|
||||
|
||||
2. In bot config:
|
||||
```javascript
|
||||
"Ai": {
|
||||
"provider": "ollama",
|
||||
"baseUrl": "http://ollama-server-ip:11434",
|
||||
"model": "llama3.2",
|
||||
}
|
||||
```
|
||||
|
||||
## Ollama Model Recommendations
|
||||
|
||||
| Model | Size | Speed | Quality | Best For |
|
||||
|-------|------|-------|---------|----------|
|
||||
| `llama3.2:3b` | 3B | Very Fast | Good | Bots needing fast responses |
|
||||
| `llama3.2:7b` | 7B | Fast | Very Good | General purpose |
|
||||
| `llama3.2:70b` | 70B | Moderate | Excellent | Smart bots, complex prompts |
|
||||
| `mistral` | 7B | Fast | Good | Balanced solution |
|
||||
| `qwen2.5:7b` | 7B | Fast | Very Good | Good instruction following |
|
||||
| `gemma2:9b` | 9B | Fast | Good | Lightweight alternative |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Refused
|
||||
|
||||
```bash
|
||||
# Check if Ollama is running
|
||||
curl http://localhost:11434/api/tags
|
||||
|
||||
# Check specific server
|
||||
curl http://192.168.1.50:11434/api/tags
|
||||
```
|
||||
|
||||
### Model Not Found
|
||||
|
||||
```bash
|
||||
# Check available models
|
||||
ollama list
|
||||
|
||||
# Pull missing model
|
||||
ollama pull llama3.2
|
||||
```
|
||||
|
||||
### JSON Parsing Errors
|
||||
|
||||
Most models support JSON mode. If issues occur:
|
||||
1. Switch to `llama3.1`, `qwen2.5`, or `mistral`
|
||||
2. Lower `temperature:` (e.g., 0.7)
|
||||
3. Increase `maxOutputTokens:` for longer responses
|
||||
|
||||
### Slow Responses
|
||||
|
||||
- Use smaller models (`llama3.2:3b` vs `70b`)
|
||||
- Increase `interval:` in config
|
||||
- Reduce `maxOutputTokens:`
|
||||
- Check network latency for remote Ollama instances
|
||||
|
||||
### Multiple Bots Overloading Ollama
|
||||
|
||||
If running many bots on one Ollama server:
|
||||
1. Use lighter models for less important bots
|
||||
2. Increase `interval:` to space requests
|
||||
3. Distribute bots across multiple Ollama instances
|
||||
|
||||
## Available Personality Prompts
|
||||
|
||||
| Personality | Description | Best Model |
|
||||
|-------------|-------------|------------|
|
||||
| `helpful` | Shy, helpful Jimmy | llama3.2, mistral |
|
||||
| `asshole` | Sarcastic, unfiltered | llama3.2:70b, gemini |
|
||||
| `Ashley` | Adult content | llama3.2, gemini |
|
||||
| `custom` | Template for custom prompts | Any |
|
||||
|
||||
## Comparing Providers
|
||||
|
||||
| Feature | Gemini | Ollama |
|
||||
|---------|--------|--------|
|
||||
| Cost | API cost | Free (local) |
|
||||
| Latency | 200-500ms | 50-500ms (local) |
|
||||
| Privacy | Cloud | 100% local |
|
||||
| Multiple Servers | No | Yes |
|
||||
| Model Choice | Limited | Any |
|
||||
| Hardware | None Required | GPU Recommended |
|
||||
| Offline | No | Yes |
|
||||
|
||||
## Command Reference
|
||||
|
||||
```bash
|
||||
/msg botname ai <personality> # Change personality
|
||||
/msg botname ai <personality> custom message # Use custom prompt
|
||||
/msg wmantly load botname Ai <personality> # Reload AI with new config
|
||||
```
|
||||
Reference in New Issue
Block a user