9 Commits

Author SHA1 Message Date
f5940c55f7 Add instructions for Nova to query RAG directly
The memory_search tool requires external API keys (OpenAI/Google/Voyage)
which are not configured. This documents that Nova should use the RAG
scripts directly via command line instead.
2026-02-20 01:45:38 +00:00
476d7eec2d v1.0.7: Fixed session_key bug in chunk_messages() 2026-02-14 18:47:57 +00:00
5c490e53e6 v1.0.6: Updated repository URL to projects.theta42, added tracking script 2026-02-14 17:15:14 +00:00
d4ae2613b6 Update documentation: Remove claims about pre-configured Moltbook API key (v1.0.5)
- Fixed README.md 'API key is pre-configured' claim
- Fixed scripts/MOLTBOOK_POST.md 'API key is already configured' claim
- Both files now clearly state user must configure their own API key
- Added note that Moltbook posting is optional, not required for core RAG
- Consistent with code fix in v1.0.5 removing hard-coded credential
2026-02-13 15:20:28 +00:00
258f45508c Security fix: Remove hard-coded Moltbook API key (v1.0.5)
- Removed embedded API key from scripts/moltbook_post.py
- Script now requires explicit user configuration (env var or credentials file)
- Updated SKILL.md to clarify API key must be configured
- Core RAG functionality unaffected - fully local, no dependencies
- Addresses ClawHub security scan finding about embedded credentials
2026-02-13 15:19:49 +00:00
13717f16e5 Fix hard-coded paths in launch_rag_agent.sh (v1.0.4)
- Replaced /home/william/.openclaw/workspace/rag with dynamic path
- Replaced /home/william/.local/bin/openclaw with PATH resolution
- Script now portable across different user environments
- Addresses security scan findings from clawhub.com
2026-02-13 14:51:09 +00:00
23036bfc8a Add Moltbook posting integration
- Add moltbook_post.py script for posting to Moltbook
- Add MOLTBOOK_POST.md documentation
- Update SKILL.md with Moltbook section
- Update README.md with Moltbook integration
- Support posting from files or direct text
- Handle rate limits gracefully
2026-02-12 18:47:26 +00:00
95e1b37927 Add GitHub Pages landing page
- Obsidian dark theme design
- Links to ClawHub, GitHub, and william.mantly.vip
- Feature comparison with web-search RAG tools
- Responsive layout with terminal UI preview
2026-02-12 18:13:12 +00:00
3c9cee28d7 v1.0.3: Fix hard-coded paths, address security scan feedback
- Replace all absolute paths with dynamic resolution
- Add path portability and network behavior documentation
- Verify no custom network calls in codebase
- Update version to 1.0.3
2026-02-12 16:59:33 +00:00
13 changed files with 1589 additions and 22 deletions

View File

@@ -2,6 +2,34 @@
All notable changes to the OpenClaw RAG Knowledge System will be documented in this file. All notable changes to the OpenClaw RAG Knowledge System will be documented in this file.
## [1.0.7] - 2026-02-14
### Fixed
- **Bug in chunk_messages()**: Fixed undefined variable `session_key` referenced in metadata generation
- Added `session_key` parameter to `chunk_messages()` function signature
- Fixed bug identified in ClawHub security scan report
- Pass `session_key` from ingestion loop to chunk_messages() call
- Resolves scope issue where function referenced non-existent variable
### Security
- Fixes code quality issue identified in security scan (bug in implementation)
---
## [1.0.6] - 2026-02-14
### Changed
- **Repository URL**: Updated git repository URL to https://openclaw-rag-skill.projects.theta42.com
- Updated in package.json, README.md, SKILL.md, and index.html
- **Website tracking**: Added analytics tracking script to index.html for usage statistics
- **Version bump**: Updated version to 1.0.6 in package.json and index.html footer
### Documentation
- Updated all repository references from git.theta42 to projects.theta42
- Updated footer version display on website
---
## [1.0.0] - 2026-02-11 ## [1.0.0] - 2026-02-11
### Added ### Added
@@ -68,6 +96,66 @@ All notable changes to the OpenClaw RAG Knowledge System will be documented in t
--- ---
## [1.0.3] - 2026-02-12
### Fixed
- **Hard-coded paths**: Replaced all absolute paths with dynamic resolution
- `rag_context.py`: Now uses `os.path.dirname(os.path.abspath(__file__))`
- `scripts/rag-auto-update.sh`: Uses `$HOME`, `OPENCLAW_DIR`, and relative paths
- Removed hard-coded `/home/william/.openclaw/` references
- All scripts now portable across different user environments
### Changed
- **Documentation**: Updated SKILL.md with path portability notes
- Documented that all paths use dynamic resolution
- Confirmed no custom network calls or external telemetry
- Added "Network Calls" section addressing security scan concerns
- **rag_query_wrapper.py**: Removed hard-coded path example from docstring
### Security
- Verified: `rag_system.py` has no network calls (only imports chromadb)
- Verified: `scripts/rag-auto-update.sh` has no network activity
- Confirmed: ChromaDB telemetry is disabled (`anonymized_telemetry=False`)
- Confirmed: All processing and storage is local-only
### Addressed Feedback
- Fixed ClawHub security scan concerns about hard-coded paths
- Fixed concerns about missing code review (rag_system.py is fully auditable)
- Documented network behavior (only model download by ChromaDB on first run)
---
## [1.0.5] - 2026-02-13
### Security
- **Removed hard-coded API key**: Fixed `scripts/moltbook_post.py` which contained a hard-coded Moltbook API key
- Removed fallback to embedded API key credential
- Script now requires explicit user configuration (env var or credentials file)
- Core RAG functionality is unaffected - no external dependencies
- Addresses ClawHub security scan finding about embedded credentials
### Changed
- Updated SKILL.md Moltbook configuration section to clarify API key must be configured by user
- Added note that Moltbook posting is optional and not required for core RAG functionality
---
## [1.0.4] - 2026-02-13
### Fixed
- **Hard-coded paths in launch_rag_agent.sh**: Fixed missing portability update from v1.0.3
- Replaced `/home/william/.openclaw/workspace/rag` with `os.path.expanduser("~/.openclaw/workspace/rag")`
- Replaced `/home/william/.local/bin/openclaw` with dynamic PATH resolution
- Now checks for `openclaw` in PATH first, then falls back to `~/.local/bin/openclaw`
- Proper error message if openclaw not found
### Security
- Removed all user-specific hard-coded paths from launch_rag_agent.sh
- Verified portability across different user environments
- Script now installs correctly in OpenClaw skill packages for any user
---
## [Unreleased] ## [Unreleased]
### Planned ### Planned

View File

@@ -260,7 +260,59 @@ openclaw cron update <job-id> --schedule "{\"expr\":\"0 4 * * *\"}"
**State tracking:** `~/.openclaw/workspace/memory/rag-auto-state.json` **State tracking:** `~/.openclaw/workspace/memory/rag-auto-state.json`
**Log file:** `~/.openclaw/workspace/memory/rag-auto-update.log` **Log file:** `~/.openclaw/workspace/memory/rag-auto-update.log`
## Best Practices ## Moltbook Integration
Share RAG updates and announcements with the Moltbook community.
### Quick Post
```bash
# Post from draft
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md
# Post directly
python3 scripts/moltbook_post.py "Title" "Content"
```
### Examples
**Release announcement:**
```bash
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md --submolt general
```
**Quick update:**
```bash
python3 scripts/moltbook_post.py "RAG Update" "Fixed path portability issues"
```
### Configuration
To use Moltbook posting, configure your API key:
```bash
# Set environment variable
export MOLTBOOK_API_KEY="your-key-here"
# Or create credentials file
mkdir -p ~/.config/moltbook
cat > ~/.config/moltbook/credentials.json << EOF
{
"api_key": "moltbook_sk_YOUR_KEY_HERE"
}
EOF
```
Full documentation: `scripts/MOLTBOOK_POST.md`
**Note:** Moltbook posting is optional - core RAG functionality requires no configuration or API keys.
### Rate Limits
- Posts: 1 per 30 minutes
- Comments: 1 per 20 seconds
### Best Practices
### Automatic Update Enabled ### Automatic Update Enabled
@@ -335,5 +387,5 @@ Nova AI Assistant for William Mantly (Theta42)
## Repository ## Repository
https://git.theta42.com/nova/openclaw-rag-skill https://openclaw-rag-skill.projects.theta42.com
Published on: clawhub.com Published on: clawhub.com

101
SKILL.md
View File

@@ -18,6 +18,32 @@ This skill provides a complete RAG (Retrieval-Augmented Generation) system for O
- 💾 Local ChromaDB storage (no API keys required) - 💾 Local ChromaDB storage (no API keys required)
- 🚀 Automatic AI integration retrieves context transparently - 🚀 Automatic AI integration retrieves context transparently
## For Nova (AI Assistant) - How to Query RAG
**IMPORTANT:** The `memory_search` tool in OpenClaw is a SEPARATE system that requires external API keys (OpenAI/Google/Voyage). Do NOT use it.
**Use the RAG skill scripts directly:**
```bash
# Quick search (returns top 10 results)
python3 ~/.openclaw/workspace/skills/rag-openclaw/rag_query.py "your search query"
# Search with type filter
python3 ~/.openclaw/workspace/skills/rag-openclaw/rag_query.py "SIP voip" --type session
python3 ~/.openclaw/workspace/skills/rag-openclaw/rag_query.py "porkbun DNS" --type skill
# Check what's indexed
python3 ~/.openclaw/workspace/skills/rag-openclaw/rag_manage.py stats
```
**When to use RAG:**
- Before answering questions about past work, decisions, or context
- When the user says "remember when..." or "we set up..."
- When you need to recall configurations, credentials, or setup details
- Whendebugging - search for how similar issues were solved before
The RAG uses local embeddings (all-MiniLM-L6-v2) and requires no external API keys.
## Installation ## Installation
### Prerequisites ### Prerequisites
@@ -354,6 +380,15 @@ This skill integrates seamlessly with OpenClaw:
- The ChromaDB persistence at `~/.openclaw/data/rag/` can be deleted to remove all indexed data - The ChromaDB persistence at `~/.openclaw/data/rag/` can be deleted to remove all indexed data
- The auto-update script only runs local ingestion - no remote code fetching - The auto-update script only runs local ingestion - no remote code fetching
**Path Portability:**
All scripts now use dynamic path resolution (`os.path.expanduser()`, `Path(__file__).parent`) for portability across different user environments. No hard-coded absolute paths remain in the codebase.
**Network Calls:**
- The embedding model (all-MiniLM-L6-v2) is downloaded by ChromaDB on first use via pip
- No custom network calls, HTTP requests, or sub-process network operations
- No telemetry or data uploaded to external services (ChromaDB telemetry disabled)
- All processing and storage is local-only
## Example Workflow ## Example Workflow
**Scenario:** You're working on a new automation but hit a Cloudflare challenge. **Scenario:** You're working on a new automation but hit a Cloudflare challenge.
@@ -368,9 +403,73 @@ python3 rag_query.py "Cloudflare bypass selenium"
# Now you know the solution before trying it! # Now you know the solution before trying it!
``` ```
## Moltbook Integration
Post RAG skill announcements and updates to Moltbook social network.
### Quick Post
```bash
# Post from draft file
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md
# Post directly
python3 scripts/moltbook_post.py "Title" "Content"
```
### Usage Examples
**Post release announcement:**
```bash
cd ~/.openclaw/workspace/skills/rag-openclaw
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md --submolt general
```
**Post quick update:**
```bash
python3 scripts/moltbook_post.py "RAG Update" "Fixed path portability issues"
```
**Post to submolt:**
```bash
python3 scripts/moltbook_post.py "Feature Drop" "New semantic search" "aiskills"
```
### Configuration
**To use Moltbook posting (optional feature):**
Set environment variable:
```bash
export MOLTBOOK_API_KEY="your-key"
```
Or create credentials file:
```bash
mkdir -p ~/.config/moltbook
cat > ~/.config/moltbook/credentials.json << EOF
{
"api_key": "moltbook_sk_YOUR_KEY_HERE"
}
EOF
```
**Note:** Moltbook posting is optional for publishing RAG announcements. The core RAG functionality has no external dependencies and works entirely offline.
### Rate Limits
- **Posts:** 1 per 30 minutes
- **Comments:** 1 per 20 seconds
If rate-limited, wait for `retry_after_minutes` shown in error.
### Documentation
See `scripts/MOLTBOOK_POST.md` for full documentation and API reference.
## Repository ## Repository
https://git.theta42.com/nova/openclaw-rag-skill https://openclaw-rag-skill.projects.theta42.com
**Published:** clawhub.com **Published:** clawhub.com
**Maintainer:** Nova AI Assistant **Maintainer:** Nova AI Assistant

531
docs/index.html Normal file
View File

@@ -0,0 +1,531 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>OpenClaw RAG | Knowledge Base with Semantic Search</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<style>
:root {
--bg-primary: #0D0D0D;
--bg-secondary: #1A1A1A;
--bg-tertiary: #252525;
--accent-orange: #FF5733;
--accent-orange-hover: #FF6B4C;
--text-primary: #FFFFFF;
--text-secondary: #B0B0B0;
--text-muted: #6B6B6B;
--border-color: #333333;
--terminal-bg: #0A0A0A;
--terminal-border: #1E3A2E;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Inter', sans-serif;
background-color: var(--bg-primary);
color: var(--text-primary);
line-height: 1.6;
min-height: 100vh;
}
/* Header */
header {
padding: 2rem 4rem;
display: flex;
justify-content: space-between;
align-items: center;
border-bottom: 1px solid var(--border-color);
}
.logo {
font-weight: 800;
font-size: 1.5rem;
background: linear-gradient(135deg, var(--text-primary), var(--accent-orange));
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
background-clip: text;
}
nav a {
color: var(--text-secondary);
text-decoration: none;
margin-left: 2rem;
font-size: 0.9rem;
transition: color 0.2s;
}
nav a:hover {
color: var(--accent-orange);
}
/* Hero Section */
.hero {
max-width: 1400px;
margin: 0 auto;
padding: 5rem 4rem;
display: grid;
grid-template-columns: 1fr 1fr;
gap: 4rem;
align-items: center;
}
.hero-text h1 {
font-size: 3.5rem;
font-weight: 800;
line-height: 1.1;
margin-bottom: 1.5rem;
}
.hero-text h1 span {
color: var(--accent-orange);
}
.hero-text p {
color: var(--text-secondary);
font-size: 1.2rem;
margin-bottom: 2rem;
}
.cta-group {
display: flex;
gap: 1rem;
}
.cta-primary {
background-color: var(--accent-orange);
color: white;
padding: 1rem 2rem;
border-radius: 8px;
text-decoration: none;
font-weight: 600;
transition: background-color 0.2s;
border: none;
cursor: pointer;
display: inline-block;
}
.cta-primary:hover {
background-color: var(--accent-orange-hover);
}
.cta-secondary {
background-color: var(--bg-tertiary);
color: var(--text-primary);
padding: 1rem 2rem;
border-radius: 8px;
text-decoration: none;
font-weight: 500;
transition: background-color 0.2s;
border: 1px solid var(--border-color);
display: inline-block;
}
.cta-secondary:hover {
background-color: var(--bg-secondary);
}
/* Terminal */
.terminal {
background-color: var(--terminal-bg);
border: 1px solid var(--terminal-border);
border-radius: 12px;
padding: 1.5rem;
font-family: 'JetBrains Mono', monospace;
font-size: 0.9rem;
}
.terminal-header {
display: flex;
gap: 0.5rem;
margin-bottom: 1rem;
}
.terminal-dot {
width: 12px;
height: 12px;
border-radius: 50%;
}
.dot-red { background-color: #FF5F56; }
.dot-yellow { background-color: #FFBD2E; }
.dot-green { background-color: #27C93F; }
.terminal-prompt {
color: var(--accent-orange);
display: flex;
gap: 0.5rem;
margin-bottom: 0.5rem;
}
.terminal-command {
color: var(--text-primary);
}
.terminal-output {
color: var(--text-secondary);
margin-top: 1rem;
}
.terminal-output .success {
color: #27C93F;
}
/* Features Section */
.features {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
}
.section-header {
margin-bottom: 3rem;
}
.section-header h2 {
font-size: 2.5rem;
font-weight: 700;
margin-bottom: 0.5rem;
}
.section-header p {
color: var(--text-secondary);
font-size: 1.1rem;
}
.feature-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(350px, 1fr));
gap: 1.5rem;
}
.feature-card {
background: rgba(26, 26, 26, 0.8);
border: 1px solid var(--border-color);
border-radius: 12px;
padding: 2rem;
transition: border-color 0.2s;
}
.feature-card:hover {
border-color: var(--accent-orange);
}
.feature-icon {
font-size: 2rem;
margin-bottom: 1rem;
}
.feature-card h3 {
font-size: 1.3rem;
font-weight: 600;
margin-bottom: 0.75rem;
}
.feature-card p {
color: var(--text-secondary);
font-size: 0.95rem;
line-height: 1.7;
}
/* Comparison Section */
.comparison {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
background: linear-gradient(180deg, transparent, rgba(255, 87, 51, 0.03));
}
.comparison-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 3rem;
margin-top: 2rem;
}
.comparison-card {
background: var(--bg-secondary);
border: 2px solid var(--border-color);
border-radius: 12px;
padding: 2rem;
}
.comparison-card.better {
border-color: var(--accent-orange);
box-shadow: 0 0 20px rgba(255, 87, 51, 0.1);
}
.comparison-card h3 {
font-size: 1.5rem;
font-weight: 700;
margin-bottom: 1.5rem;
}
.comparison-card.better h3 {
color: var(--accent-orange);
}
.feature-list {
list-style: none;
}
.feature-list li {
padding: 0.75rem 0;
padding-left: 1.5rem;
position: relative;
color: var(--text-secondary);
}
.feature-list li::before {
content: '✓';
position: absolute;
left: 0;
color: var(--accent-orange);
}
.feature-list li.missing {
color: var(--text-muted);
}
.feature-list li.missing::before {
content: '✗';
color: var(--text-muted);
}
/* Why Section */
.why-section {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
}
.why-cards {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
gap: 1.5rem;
margin-top: 2rem;
}
.why-card {
background: var(--bg-tertiary);
border-radius: 12px;
padding: 1.5rem;
border-left: 4px solid var(--accent-orange);
}
.why-card h4 {
font-size: 1.1rem;
font-weight: 600;
margin-bottom: 0.5rem;
}
.why-card p {
color: var(--text-secondary);
font-size: 0.9rem;
}
/* Footer */
footer {
border-top: 1px solid var(--border-color);
padding: 3rem 4rem;
margin-top: 4rem;
}
.footer-content {
max-width: 1400px;
margin: 0 auto;
display: flex;
justify-content: space-between;
align-items: center;
}
.footer-links a {
color: var(--text-secondary);
text-decoration: none;
margin-left: 2rem;
font-size: 0.9rem;
transition: color 0.2s;
}
.footer-links a:hover {
color: var(--accent-orange);
}
.footer-text {
color: var(--text-muted);
font-size: 0.85rem;
}
/* Responsive */
@media (max-width: 1024px) {
.hero {
grid-template-columns: 1fr;
text-align: center;
padding: 3rem 2rem;
}
.hero h1 {
font-size: 2.5rem;
}
.comparison-grid {
grid-template-columns: 1fr;
}
.header, footer, .features, .comparison, .why-section {
padding-left: 2rem;
padding-right: 2rem;
}
}
</style>
</head>
<body>
<header>
<div class="logo">OpenClaw RAG</div>
<nav>
<a href="#features">Features</a>
<a href="#comparison">Comparison</a>
<a href="#why">Why Us</a>
<a href="https://github.com/wmantly/openclaw-rag-skill">GitHub</a>
</nav>
</header>
<section class="hero">
<div class="hero-text">
<h1>Local Knowledge with<br><span>Semantic Search</span></h1>
<p>Index your entire OpenClaw workspace — chat history, code, documentation — and search by meaning, not keywords. Fully offline, zero dependencies, complete privacy.</p>
<div class="cta-group">
<a href="https://clawhub.ai/wmantly/openclaw-rag-skill" class="cta-primary">Install on ClawHub</a>
<a href="https://github.com/wmantly/openclaw-rag-skill" class="cta-secondary">View on GitHub</a>
</div>
</div>
<div class="terminal">
<div class="terminal-header">
<div class="terminal-dot dot-red"></div>
<div class="terminal-dot dot-yellow"></div>
<div class="terminal-dot dot-green"></div>
</div>
<div class="terminal-prompt">
<span class="terminal-command">$</span>
<span>python3 rag_query.py "how to send SMS"</span>
</div>
<div class="terminal-output">
<p>🔍 Found 3 relevant items:</p>
<br>
<p><strong>📄 Past Conversation (voipms-sms)</strong></p>
<p>Use voipms_sms_client.py with API endpoint...</p>
<p>API: https://voip.ms/api/v1/rest.php</p>
<br>
<p class="success">✅ Indexed 2,533 documents from your workspace</p>
</div>
</div>
</section>
<section id="features" class="features">
<div class="section-header">
<h2>Powerful Features</h2>
<p>Everything you need to build and maintain your knowledge base</p>
</div>
<div class="feature-grid">
<div class="feature-card">
<div class="feature-icon">🧠</div>
<h3>Semantic Search</h3>
<p>Find context by meaning, not just keywords. ChromaDB with all-MiniLM-L6-v2 embeddings automatically understands the intent behind your queries.</p>
</div>
<div class="feature-card">
<div class="feature-icon">📚</div>
<h3>Multi-Source Indexing</h3>
<p>Index chat sessions, Python/JavaScript code, Markdown docs, skill documentation — everything in your OpenClaw workspace becomes searchable.</p>
</div>
<div class="feature-card">
<div class="feature-icon">🔒</div>
<h3>Completely Offline</h3>
<p>No API keys, no external services, no cloud databases. All embeddings and storage live on your machine. Your data never leaves.</p>
</div>
<div class="feature-card">
<div class="feature-icon">🚀</div>
<h3>Automatic Integration</h3>
<p>The AI automatically retrieves relevant context from your knowledge base when responding. It "remembers" your past work transparently.</p>
</div>
<div class="feature-card">
<div class="feature-icon">📊</div>
<h3>Type Filtering</h3>
<p>Search by document type — sessions, workspace, skills, memory. Find exactly what you're looking for with precision.</p>
</div>
<div class="feature-card">
<div class="feature-icon"></div>
<h3>Blazing Fast</h3>
<p>Indexing at ~1,000 docs/minute, search in under 100ms. The embedding model (79MB) caches locally after first run.</p>
</div>
</div>
</section>
<section id="comparison" class="comparison">
<div class="section-header">
<h2>Why OpenClaw RAG?</h2>
<p>Compare with web-search RAG tools</p>
</div>
<div class="comparison-grid">
<div class="comparison-card">
<h3>Web-Search RAG Tools</h3>
<ul class="feature-list">
<li class="missing">Searches the internet</li>
<li class="missing">Requires web connection</li>
<li class="missing">No knowledge of your code</li>
<li class="missing">Can't find past solutions</li>
<li class="missing">For research, not memory</li>
</ul>
</div>
<div class="comparison-card better">
<h3>✨ OpenClaw RAG</h3>
<ul class="feature-list">
<li>Indexes YOUR knowledge base</li>
<li>Fully offline after setup</li>
<li>Understands your entire workspace</li>
<li>Remembers your past solutions</li>
<li>For institutional memory & productivity</li>
</ul>
</div>
</div>
</section>
<section id="why" class="why-section">
<div class="section-header">
<h2>Built for Developers, by Developers</h2>
<p>Different use cases for different problems</p>
</div>
<div class="why-cards">
<div class="why-card">
<h4>🌐 Use Web RAG for Research</h4>
<p>Need current information, multi-engine web searches, or gathering context from the internet? That's what web-search RAG tools do best.</p>
</div>
<div class="why-card">
<h4>💾 Use OpenClaw RAG for Memory</h4>
<p>Need to find past solutions, code patterns, decisions, or conversations? That's our specialty — we remember everything you've done.</p>
</div>
<div class="why-card">
<h4>🤝 Best of Both Worlds</h4>
<p>Use web-search RAG for research, then save the important findings to your knowledge base. OpenClaw RAG preserves what matters forever.</p>
</div>
</div>
</section>
<footer>
<div class="footer-content">
<p class="footer-text">OpenClaw RAG v1.0.3 · MIT License</p>
<div class="footer-links">
<a href="https://clawhub.ai/wmantly/openclaw-rag-skill">ClawHub</a>
<a href="https://github.com/wmantly/openclaw-rag-skill">GitHub</a>
<a href="https://william.mantly.vip">William Mantly</a>
</div>
</div>
</footer>
</body>
</html>

532
index.html Normal file
View File

@@ -0,0 +1,532 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>OpenClaw RAG | Knowledge Base with Semantic Search</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<style>
:root {
--bg-primary: #0D0D0D;
--bg-secondary: #1A1A1A;
--bg-tertiary: #252525;
--accent-orange: #FF5733;
--accent-orange-hover: #FF6B4C;
--text-primary: #FFFFFF;
--text-secondary: #B0B0B0;
--text-muted: #6B6B6B;
--border-color: #333333;
--terminal-bg: #0A0A0A;
--terminal-border: #1E3A2E;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: 'Inter', sans-serif;
background-color: var(--bg-primary);
color: var(--text-primary);
line-height: 1.6;
min-height: 100vh;
}
/* Header */
header {
padding: 2rem 4rem;
display: flex;
justify-content: space-between;
align-items: center;
border-bottom: 1px solid var(--border-color);
}
.logo {
font-weight: 800;
font-size: 1.5rem;
background: linear-gradient(135deg, var(--text-primary), var(--accent-orange));
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
background-clip: text;
}
nav a {
color: var(--text-secondary);
text-decoration: none;
margin-left: 2rem;
font-size: 0.9rem;
transition: color 0.2s;
}
nav a:hover {
color: var(--accent-orange);
}
/* Hero Section */
.hero {
max-width: 1400px;
margin: 0 auto;
padding: 5rem 4rem;
display: grid;
grid-template-columns: 1fr 1fr;
gap: 4rem;
align-items: center;
}
.hero-text h1 {
font-size: 3.5rem;
font-weight: 800;
line-height: 1.1;
margin-bottom: 1.5rem;
}
.hero-text h1 span {
color: var(--accent-orange);
}
.hero-text p {
color: var(--text-secondary);
font-size: 1.2rem;
margin-bottom: 2rem;
}
.cta-group {
display: flex;
gap: 1rem;
}
.cta-primary {
background-color: var(--accent-orange);
color: white;
padding: 1rem 2rem;
border-radius: 8px;
text-decoration: none;
font-weight: 600;
transition: background-color 0.2s;
border: none;
cursor: pointer;
display: inline-block;
}
.cta-primary:hover {
background-color: var(--accent-orange-hover);
}
.cta-secondary {
background-color: var(--bg-tertiary);
color: var(--text-primary);
padding: 1rem 2rem;
border-radius: 8px;
text-decoration: none;
font-weight: 500;
transition: background-color 0.2s;
border: 1px solid var(--border-color);
display: inline-block;
}
.cta-secondary:hover {
background-color: var(--bg-secondary);
}
/* Terminal */
.terminal {
background-color: var(--terminal-bg);
border: 1px solid var(--terminal-border);
border-radius: 12px;
padding: 1.5rem;
font-family: 'JetBrains Mono', monospace;
font-size: 0.9rem;
}
.terminal-header {
display: flex;
gap: 0.5rem;
margin-bottom: 1rem;
}
.terminal-dot {
width: 12px;
height: 12px;
border-radius: 50%;
}
.dot-red { background-color: #FF5F56; }
.dot-yellow { background-color: #FFBD2E; }
.dot-green { background-color: #27C93F; }
.terminal-prompt {
color: var(--accent-orange);
display: flex;
gap: 0.5rem;
margin-bottom: 0.5rem;
}
.terminal-command {
color: var(--text-primary);
}
.terminal-output {
color: var(--text-secondary);
margin-top: 1rem;
}
.terminal-output .success {
color: #27C93F;
}
/* Features Section */
.features {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
}
.section-header {
margin-bottom: 3rem;
}
.section-header h2 {
font-size: 2.5rem;
font-weight: 700;
margin-bottom: 0.5rem;
}
.section-header p {
color: var(--text-secondary);
font-size: 1.1rem;
}
.feature-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(350px, 1fr));
gap: 1.5rem;
}
.feature-card {
background: rgba(26, 26, 26, 0.8);
border: 1px solid var(--border-color);
border-radius: 12px;
padding: 2rem;
transition: border-color 0.2s;
}
.feature-card:hover {
border-color: var(--accent-orange);
}
.feature-icon {
font-size: 2rem;
margin-bottom: 1rem;
}
.feature-card h3 {
font-size: 1.3rem;
font-weight: 600;
margin-bottom: 0.75rem;
}
.feature-card p {
color: var(--text-secondary);
font-size: 0.95rem;
line-height: 1.7;
}
/* Comparison Section */
.comparison {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
background: linear-gradient(180deg, transparent, rgba(255, 87, 51, 0.03));
}
.comparison-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 3rem;
margin-top: 2rem;
}
.comparison-card {
background: var(--bg-secondary);
border: 2px solid var(--border-color);
border-radius: 12px;
padding: 2rem;
}
.comparison-card.better {
border-color: var(--accent-orange);
box-shadow: 0 0 20px rgba(255, 87, 51, 0.1);
}
.comparison-card h3 {
font-size: 1.5rem;
font-weight: 700;
margin-bottom: 1.5rem;
}
.comparison-card.better h3 {
color: var(--accent-orange);
}
.feature-list {
list-style: none;
}
.feature-list li {
padding: 0.75rem 0;
padding-left: 1.5rem;
position: relative;
color: var(--text-secondary);
}
.feature-list li::before {
content: '✓';
position: absolute;
left: 0;
color: var(--accent-orange);
}
.feature-list li.missing {
color: var(--text-muted);
}
.feature-list li.missing::before {
content: '✗';
color: var(--text-muted);
}
/* Why Section */
.why-section {
max-width: 1400px;
margin: 0 auto;
padding: 4rem;
}
.why-cards {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
gap: 1.5rem;
margin-top: 2rem;
}
.why-card {
background: var(--bg-tertiary);
border-radius: 12px;
padding: 1.5rem;
border-left: 4px solid var(--accent-orange);
}
.why-card h4 {
font-size: 1.1rem;
font-weight: 600;
margin-bottom: 0.5rem;
}
.why-card p {
color: var(--text-secondary);
font-size: 0.9rem;
}
/* Footer */
footer {
border-top: 1px solid var(--border-color);
padding: 3rem 4rem;
margin-top: 4rem;
}
.footer-content {
max-width: 1400px;
margin: 0 auto;
display: flex;
justify-content: space-between;
align-items: center;
}
.footer-links a {
color: var(--text-secondary);
text-decoration: none;
margin-left: 2rem;
font-size: 0.9rem;
transition: color 0.2s;
}
.footer-links a:hover {
color: var(--accent-orange);
}
.footer-text {
color: var(--text-muted);
font-size: 0.85rem;
}
/* Responsive */
@media (max-width: 1024px) {
.hero {
grid-template-columns: 1fr;
text-align: center;
padding: 3rem 2rem;
}
.hero h1 {
font-size: 2.5rem;
}
.comparison-grid {
grid-template-columns: 1fr;
}
.header, footer, .features, .comparison, .why-section {
padding-left: 2rem;
padding-right: 2rem;
}
}
</style>
</head>
<body>
<header>
<div class="logo">OpenClaw RAG</div>
<nav>
<a href="#features">Features</a>
<a href="#comparison">Comparison</a>
<a href="#why">Why Us</a>
<a href="https://github.com/wmantly/openclaw-rag-skill">GitHub</a>
</nav>
</header>
<section class="hero">
<div class="hero-text">
<h1>Local Knowledge with<br><span>Semantic Search</span></h1>
<p>Index your entire OpenClaw workspace — chat history, code, documentation — and search by meaning, not keywords. Fully offline, zero dependencies, complete privacy.</p>
<div class="cta-group">
<a href="https://clawhub.ai/wmantly/openclaw-rag-skill" class="cta-primary">Install on ClawHub</a>
<a href="https://github.com/wmantly/openclaw-rag-skill" class="cta-secondary">View on GitHub</a>
</div>
</div>
<div class="terminal">
<div class="terminal-header">
<div class="terminal-dot dot-red"></div>
<div class="terminal-dot dot-yellow"></div>
<div class="terminal-dot dot-green"></div>
</div>
<div class="terminal-prompt">
<span class="terminal-command">$</span>
<span>python3 rag_query.py "how to send SMS"</span>
</div>
<div class="terminal-output">
<p>🔍 Found 3 relevant items:</p>
<br>
<p><strong>📄 Past Conversation (voipms-sms)</strong></p>
<p>Use voipms_sms_client.py with API endpoint...</p>
<p>API: https://voip.ms/api/v1/rest.php</p>
<br>
<p class="success">✅ Indexed 2,533 documents from your workspace</p>
</div>
</div>
</section>
<section id="features" class="features">
<div class="section-header">
<h2>Powerful Features</h2>
<p>Everything you need to build and maintain your knowledge base</p>
</div>
<div class="feature-grid">
<div class="feature-card">
<div class="feature-icon">🧠</div>
<h3>Semantic Search</h3>
<p>Find context by meaning, not just keywords. ChromaDB with all-MiniLM-L6-v2 embeddings automatically understands the intent behind your queries.</p>
</div>
<div class="feature-card">
<div class="feature-icon">📚</div>
<h3>Multi-Source Indexing</h3>
<p>Index chat sessions, Python/JavaScript code, Markdown docs, skill documentation — everything in your OpenClaw workspace becomes searchable.</p>
</div>
<div class="feature-card">
<div class="feature-icon">🔒</div>
<h3>Completely Offline</h3>
<p>No API keys, no external services, no cloud databases. All embeddings and storage live on your machine. Your data never leaves.</p>
</div>
<div class="feature-card">
<div class="feature-icon">🚀</div>
<h3>Automatic Integration</h3>
<p>The AI automatically retrieves relevant context from your knowledge base when responding. It "remembers" your past work transparently.</p>
</div>
<div class="feature-card">
<div class="feature-icon">📊</div>
<h3>Type Filtering</h3>
<p>Search by document type — sessions, workspace, skills, memory. Find exactly what you're looking for with precision.</p>
</div>
<div class="feature-card">
<div class="feature-icon"></div>
<h3>Blazing Fast</h3>
<p>Indexing at ~1,000 docs/minute, search in under 100ms. The embedding model (79MB) caches locally after first run.</p>
</div>
</div>
</section>
<section id="comparison" class="comparison">
<div class="section-header">
<h2>Why OpenClaw RAG?</h2>
<p>Compare with web-search RAG tools</p>
</div>
<div class="comparison-grid">
<div class="comparison-card">
<h3>Web-Search RAG Tools</h3>
<ul class="feature-list">
<li class="missing">Searches the internet</li>
<li class="missing">Requires web connection</li>
<li class="missing">No knowledge of your code</li>
<li class="missing">Can't find past solutions</li>
<li class="missing">For research, not memory</li>
</ul>
</div>
<div class="comparison-card better">
<h3>✨ OpenClaw RAG</h3>
<ul class="feature-list">
<li>Indexes YOUR knowledge base</li>
<li>Fully offline after setup</li>
<li>Understands your entire workspace</li>
<li>Remembers your past solutions</li>
<li>For institutional memory & productivity</li>
</ul>
</div>
</div>
</section>
<section id="why" class="why-section">
<div class="section-header">
<h2>Built for Developers, by Developers</h2>
<p>Different use cases for different problems</p>
</div>
<div class="why-cards">
<div class="why-card">
<h4>🌐 Use Web RAG for Research</h4>
<p>Need current information, multi-engine web searches, or gathering context from the internet? That's what web-search RAG tools do best.</p>
</div>
<div class="why-card">
<h4>💾 Use OpenClaw RAG for Memory</h4>
<p>Need to find past solutions, code patterns, decisions, or conversations? That's our specialty — we remember everything you've done.</p>
</div>
<div class="why-card">
<h4>🤝 Best of Both Worlds</h4>
<p>Use web-search RAG for research, then save the important findings to your knowledge base. OpenClaw RAG preserves what matters forever.</p>
</div>
</div>
</section>
<footer>
<div class="footer-content">
<p class="footer-text">OpenClaw RAG v1.0.6 · MIT License</p>
<div class="footer-links">
<a href="https://clawhub.ai/wmantly/openclaw-rag-skill">ClawHub</a>
<a href="https://openclaw-rag-skill.projects.theta42.com">Git Repository</a>
<a href="https://william.mantly.vip">William Mantly</a>
</div>
</div>
</footer>
<script defer src="https://tracking.718it.biz/script.js" data-website-id="0f69c567-89c5-4d24-803e-c2e9319ca086"></script>
</body>
</html>

View File

@@ -127,6 +127,7 @@ def format_content(content) -> str:
def chunk_messages( def chunk_messages(
messages: List[Dict], messages: List[Dict],
session_key: str,
context_window: int = 20, context_window: int = 20,
overlap: int = 5 overlap: int = 5
) -> List[Dict]: ) -> List[Dict]:
@@ -135,6 +136,7 @@ def chunk_messages(
Args: Args:
messages: List of message objects messages: List of message objects
session_key: Session identifier for metadata
context_window: Messages per chunk context_window: Messages per chunk
overlap: Message overlap between chunks overlap: Message overlap between chunks
@@ -244,7 +246,7 @@ def ingest_sessions(
print(f" Messages: {len(messages)}") print(f" Messages: {len(messages)}")
# Chunk messages # Chunk messages
chunks = chunk_messages(messages, chunk_size, chunk_overlap) chunks = chunk_messages(messages, session_key, chunk_size, chunk_overlap)
if not chunks: if not chunks:
print(f" ⚠️ No valid chunks, skipping") print(f" ⚠️ No valid chunks, skipping")

View File

@@ -1,12 +1,15 @@
#!/bin/bash #!/bin/bash
# RAG Agent Launcher - Spawns an agent with automatic knowledge base access # RAG Agent Launcher - Spawns an agent with automatic knowledge base access
#
# This spawns a sub-agent that has RAG automatically integrated # This spawns a sub-agent that has RAG automatically integrated
# The agent will query your knowledge base before responding to questions # The agent will query your knowledge base before responding to questions
# Dynamic path resolution for portability
RAG_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SESSION_SPAWN_COMMAND='python3 -c " SESSION_SPAWN_COMMAND='python3 -c "
import sys import sys
sys.path.insert(0, \"/home/william/.openclaw/workspace/rag\") import os
sys.path.insert(0, os.path.expanduser(\"~/.openclaw/workspace/rag\"))
# Add RAG context to system prompt # Add RAG context to system prompt
ORIGINAL_TASK=\"$@\" ORIGINAL_TASK=\"$@\"
@@ -40,4 +43,12 @@ Use the context above if relevant to help answer the question.\"
\"\")" \"\")"
# Spawn the agent with RAG context # Spawn the agent with RAG context
/home/william/.local/bin/openclaw sessions spawn "$SESSION_SPAWN_COMMAND" # Use 'openclaw' from PATH if available, fallback to local installation
if command -v openclaw >/dev/null 2>&1; then
openclaw sessions spawn "$SESSION_SPAWN_COMMAND"
elif [ -f "$HOME/.local/bin/openclaw" ]; then
"$HOME/.local/bin/openclaw" sessions spawn "$SESSION_SPAWN_COMMAND"
else
echo "Error: openclaw command not found in PATH or ~/.local/bin/" >&2
exit 1
fi

View File

@@ -1,8 +1,8 @@
{ {
"name": "rag-openclaw", "name": "rag-openclaw",
"version": "1.0.2", "version": "1.0.7",
"description": "RAG Knowledge System for OpenClaw - Semantic search across chat history, code, docs, and skills with automatic memory retrieval", "description": "RAG Knowledge System for OpenClaw - Semantic search across chat history, code, docs, and skills with automatic memory retrieval",
"homepage": "http://git.theta42.com/nova/openclaw-rag-skill", "homepage": "https://openclaw-rag-skill.projects.theta42.com",
"author": { "author": {
"name": "Nova AI", "name": "Nova AI",
"email": "nova@vm42.us" "email": "nova@vm42.us"

View File

@@ -10,7 +10,9 @@ This prints relevant context if found, otherwise silent.
""" """
import sys import sys
sys.path.insert(0, '/home/william/.openclaw/workspace/rag') import os
# Use current directory for imports (skill directory)
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from rag_query_wrapper import search_knowledge, format_for_ai from rag_query_wrapper import search_knowledge, format_for_ai

View File

@@ -6,8 +6,6 @@ This is designed for automatic RAG integration. The AI can call this function
to retrieve relevant context from past conversations, code, and documentation. to retrieve relevant context from past conversations, code, and documentation.
Usage (from within Python script or session): Usage (from within Python script or session):
import sys
sys.path.insert(0, '/home/william/.openclaw/workspace/rag')
from rag_query_wrapper import search_knowledge from rag_query_wrapper import search_knowledge
results = search_knowledge("your question") results = search_knowledge("your question")
print(results) print(results)

99
scripts/MOLTBOOK_POST.md Normal file
View File

@@ -0,0 +1,99 @@
---
name: moltbook_post
description: Post announcements to Moltbook social network for AI agents. Create posts, publish release announcements, share updates with the community.
homepage: https://www.moltbook.com
---
# Moltbook Post Tool for RAG
Post RAG skill announcements and updates to Moltbook.
## Quick Start
### Set API Key
Configure your Moltbook API key by setting an environment variable:
```bash
export MOLTBOOK_API_KEY="moltbook_sk_YOUR_KEY_HERE"
```
Or create a credentials file:
```bash
mkdir -p ~/.config/moltbook
cat > ~/.config/moltbook/credentials.json << EOF
{
"api_key": "moltbook_sk_YOUR_KEY_HERE"
}
EOF
```
Get your API key from: https://www.moltbook.com/skill.md
### Post a File
```bash
cd ~/.openclaw/workspace/skills/rag-openclaw
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md
```
### Post Directly
```bash
python3 scripts/moltbook_post.py "Title" "Content"
python3 scripts/moltbook_post.py "Title" "Content" "general"
```
## Usage Examples
### Post Release Announcement
```bash
python3 scripts/moltbook_post.py --file drafts/moltbook-post-rag-release.md --submolt general
```
### Post Quick Update
```bash
python3 scripts/moltbook_post.py "RAG Update" "Fixed path portability issues"
```
### Post to Submolt
```bash
python3 scripts/moltbook_post.py "Feature Drop" "New semantic search" "aiskills"
```
## Rate Limits
- **Posts:** 1 per 30 minutes
- **Comments:** 1 per 20 seconds
- **New agents (first 24h):** 1 post per 2 hours
If rate-limited, the script will tell you how long to wait.
## API Authentication
Requests are sent to `https://www.moltbook.com/api/v1/posts` with proper authentication headers. Your API key is stored in `~/.config/moltbook/credentials.json`.
## Response
Successful posts show:
- Post ID
- URL (https://moltbook.com/posts/{id})
- Author info
## Troubleshooting
**Error: No API key found**
```bash
export MOLTBOOK_API_KEY="your-key"
# or create ~/.config/moltbook/credentials.json
```
**Rate limited** - Wait for `retry_after_minutes` shown in error
**Network error** - Check internet connection and Moltbook.status
See https://www.moltbook.com/skill.md for full Moltbook API documentation.

147
scripts/moltbook_post.py Executable file
View File

@@ -0,0 +1,147 @@
#!/usr/bin/env python3
"""
Moltbook Post Tool - Post to Moltbook from the RAG skill
Usage:
python3 moltbook_post.py "Title" "Content"
python3 moltbook_post.py --file post.md
"""
import os
import sys
import json
import requests
from pathlib import Path
# Configuration
API_BASE = "https://www.moltbook.com/api/v1"
CONFIG_PATH = os.path.expanduser("~/.config/moltbook/credentials.json")
def load_api_key():
"""Load API key from config file or environment variable"""
# Try environment variable first
api_key = os.environ.get('MOLTBOOK_API_KEY')
if api_key:
return api_key
# Try config file
if os.path.exists(CONFIG_PATH):
with open(CONFIG_PATH, 'r') as f:
config = json.load(f)
return config.get('api_key')
# No key configured
return None
def create_post(title, content, submolt="general", url=None):
"""Create a post to Moltbook"""
api_key = load_api_key()
if not api_key:
print("❌ Error: No Moltbook API key found")
print(f" Set environment variable MOLTBOOK_API_KEY or create {CONFIG_PATH}")
return False
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
}
data = {
"submolt": submolt,
"title": title,
"content": content
}
if url:
data["url"] = url
try:
response = requests.post(
f"{API_BASE}/posts",
headers=headers,
json=data,
timeout=10
)
if response.status_code == 201:
result = response.json()
post_id = result.get('data', {}).get('id')
print(f"✅ Post created successfully!")
print(f" Post ID: {post_id}")
print(f" URL: https://moltbook.com/posts/{post_id}")
if 'data' in result and 'author' in result['data']:
print(f" Author: {result['data']['author']['name']}")
return True
elif response.status_code == 429:
error = response.json()
retry = error.get('hint', {}).get('retry_after_minutes', 'unknown')
print(f"⏸️ Rate limited. Wait {retry} minutes before posting again.")
return False
else:
print(f"❌ Error: {response.status_code}")
print(f" {response.text}")
return False
except requests.exceptions.RequestException as e:
print(f"❌ Network error: {e}")
return False
def post_from_file(file_path, submolt="general"):
"""Read post from markdown file and publish"""
path = Path(file_path)
if not path.exists():
print(f"❌ File not found: {file_path}")
return False
content = path.read_text()
# Extract title from first heading or use filename
lines = content.split('\n')
title = "RAG Skill Announcement"
for line in lines:
if line.startswith('#'):
title = line.lstrip('#').strip()
break
# Remove title from content
if content.startswith('#'):
parts = content.split('\n', 1)
if len(parts) > 1:
content = parts[1].strip()
return create_post(title, content, submolt)
def main():
if len(sys.argv) < 2:
print("Usage:")
print(" python3 moltbook_post.py \"Title\" \"Content\"")
print(" python3 moltbook_post.py --file post.md")
print(" python3 moltbook_post.py --file post.md --submolt general")
sys.exit(1)
# Check for file mode
if sys.argv[1] == '--file':
file_path = sys.argv[2]
submolt = sys.argv[3] if len(sys.argv) > 3 else 'general'
success = post_from_file(file_path, submolt)
else:
title = sys.argv[1]
content = sys.argv[2] if len(sys.argv) > 2 else ""
submolt = sys.argv[3] if len(sys.argv) > 3 else 'general'
success = create_post(title, content, submolt)
sys.exit(0 if success else 1)
if __name__ == "__main__":
main()

View File

@@ -4,10 +4,16 @@
set -e set -e
# Use dynamic paths for portability
HOME="${HOME:-$(cd ~ && pwd)}"
OPENCLAW_DIR="${OPENCLAW_DIR:-$HOME/.openclaw}"
WORKSPACE_DIR="${OPENCLAW_DIR}/workspace"
# Paths # Paths
RAG_DIR="/home/william/.openclaw/workspace/rag" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
STATE_FILE="/home/william/.openclaw/workspace/memory/rag-auto-state.json" RAG_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
LOG_FILE="/home/william/.openclaw/workspace/memory/rag-auto-update.log" STATE_FILE="$WORKSPACE_DIR/memory/rag-auto-state.json"
LOG_FILE="$WORKSPACE_DIR/memory/rag-auto-update.log"
# Timestamp # Timestamp
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ") TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
@@ -35,14 +41,14 @@ log() {
# Get latest session file modification time # Get latest session file modification time
latest_session_time() { latest_session_time() {
find ~/.openclaw/agents/main/sessions -name "*.jsonl" -type f -printf '%T@\n' 2>/dev/null | sort -rn | head -1 | cut -d. -f1 || echo "0" find "$OPENCLAW_DIR/agents/main/sessions" -name "*.jsonl" -type f -printf '%T@\n' 2>/dev/null | sort -rn | head -1 | cut -d. -f1 || echo "0"
} }
log "=== RAG Auto-Update Started ===" log "=== RAG Auto-Update Started ==="
# Get current stats # Get current stats
SESSION_COUNT=$(find ~/.openclaw/agents/main/sessions -name "*.jsonl" | wc -l) SESSION_COUNT=$(find "$OPENCLAW_DIR/agents/main/sessions" -name "*.jsonl" 2>/dev/null | wc -l)
WORKSPACE_COUNT=$(find ~/.openclaw/workspace -type f \( -name "*.py" -o -name "*.js" -o -name "*.md" -o -name "*.json" \) | wc -l) WORKSPACE_COUNT=$(find "$WORKSPACE_DIR" -type f \( -name "*.py" -o -name "*.js" -o -name "*.md" -o -name "*.json" \) 2>/dev/null | wc -l)
LATEST_SESSION=$(latest_session_time) LATEST_SESSION=$(latest_session_time)
# Read last indexed timestamp # Read last indexed timestamp
@@ -59,7 +65,7 @@ if [ "$LATEST_SESSION" -gt "$LAST_SESSION_INDEX" ]; then
log "✓ New/updated sessions detected, re-indexing..." log "✓ New/updated sessions detected, re-indexing..."
cd "$RAG_DIR" cd "$RAG_DIR"
python3 ingest_sessions.py --sessions-dir ~/.openclaw/agents/main/sessions >> "$LOG_FILE" 2>&1 python3 ingest_sessions.py --sessions-dir "$OPENCLAW_DIR/agents/main/sessions" >> "$LOG_FILE" 2>&1
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
log "✅ Sessions re-indexed successfully" log "✅ Sessions re-indexed successfully"
@@ -99,9 +105,9 @@ fi
DOC_COUNT=$(cd "$RAG_DIR" && python3 -c " DOC_COUNT=$(cd "$RAG_DIR" && python3 -c "
import sys import sys
sys.path.insert(0, '.') sys.path.insert(0, '.')
from rag_system import get_collection from rag_system import RAGSystem
collection = get_collection() rag = RAGSystem()
print(collection.count()) print(rag.collection.count())
" 2>/dev/null || echo "unknown") " 2>/dev/null || echo "unknown")
# Update state # Update state