# Storage/Trade Bot System Documentation ## Overview A plugin-based storage and trading system for Minecraft bots. Any bot (`ez` or others) equipped with the StoragePlugin can: - Automatically discover and track chests in a storage area - Sort incoming items into shulker boxes - Track full inventory metadata (NBT, enchantments, durability) - Provide web API for inventory viewing (24/7) - Handle deposit/withdraw via the `/trade` command **Key Design Principle**: The system is NOT hardcoded to any specific bot name. Any bot can be configured with the StoragePlugin via the plugin system. --- ## Requirements ### Functional Requirements #### 1. Chest Discovery - Scan all chests within configurable render distance - Automatically detect single vs double chests - Assign row/column positions for organization - No signs required - chests are positional #### 2. Storage Organization - **Only shulker boxes stored in chests** - no loose items - **One item type per shulker** - no mixing items in a shulker - Automatic categorization of items (minerals, food, tools, etc.) - Unlimited empty shulkers available from reserve #### 3. Trade Integration - Use existing `/trade` command for all item transfers - Max 12 slots per trade (server limitation) - Deposit flow: player trades → bot sorts → items stored - Withdraw flow: player requests → bot gathers → trade window #### 4. Database Persistence - SQLite database for inventory tracking - Database independent of bot being online - Web API reads directly from database (24/7 availability) #### 5. Permission System - Database-driven permissions (no file editing) - Roles: `owner`, `team`, `readonly` - Commands and web access limited by role #### 6. Web Interface - Simple but fully functional UI - Search inventory - View item counts and locations - Request withdrawals via web - No login required (whisper challenge for auth) ### Technical Requirements #### Stack - Node.js + mineflayer (existing infrastructure) - SQLite for database (via `sqlite3` npm package) - Express for web API - Minecraft server: CoreJourney (existing) #### Constraints - Plain shulker boxes only (no dyes, no NBT names) - Trade window max 12 slots - Bot location is secret (no player access to chests) - Web server must run 24/7 (separate from bot process) --- ## Architecture ### System Diagram ``` ┌─────────────────────────────────────────────────────────────┐ │ GAME LAYER │ ├─────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────┐ ┌─────────────┐ │ │ │ StorageBot │ (Optional) │ Other Bots │ │ │ │ (e.g., ez) │ │ │ │ │ │ - Scan │ │ - Proxy │ │ │ │ - Store │ │ - Messages │ │ │ │ - Trade │ │ │ │ │ └──────┬──────┘ └─────────────┘ │ │ │ │ │ │ Commands, Trade Events │ │ ▼ │ │ ┌────────────────────────┐ │ │ │ StoragePlugin │ ← ANY bot can use this │ │ │ (Business Logic) │ via plugin system │ │ └────────────┬───────────┘ │ └────────────────┼──────────────────────────────────────────────┘ │ ┌────────────────┼──────────────────────────────────────────────┐ │ DATABASE LAYER │ ├─────────────────────────────────────────────────────────────┤ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ permissions │ │ chests │ │ shulkers │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ shulker_items│ │ trades │ │ item_index │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └────────────────┼──────────────────────────────────────────────┘ │ (SQLite: ./storage/storage.db) ┌────────────────┼──────────────────────────────────────────────┐ │ WEB LAYER (24/7) │ ├─────────────────────────────────────────────────────────────┤ │ ┌──────────────┐ ┌──────────────┐ │ │ │ Express │────────▶│ Web UI │ │ │ │ Server │ API │ (HTML/JS) │ │ │ └──────────────┘ └──────────────┘ │ │ ▲ │ │ │ REST API │ │ │ │ │ /api/inventory, /api/chests, /api/withdraw, ... │ └─────────────────────────────────────────────────────────────┘ ``` ### Plugin Structure (Bot-Agnostic) ```javascript // Configuration in conf/secrets.js "mc": { "bots": { "ez": { "plugins": { "Storage": { // Bot can be swapped anytime } } }, // Another bot can use Storage plugin: "art": { "plugins": { "Storage": { // Different location, same functionality } } } } } ``` --- ## Database Schema ### Tables #### `permissions` Manage access to the storage system. ```sql CREATE TABLE permissions ( id INTEGER PRIMARY KEY AUTOINCREMENT, player_name TEXT UNIQUE NOT NULL, role TEXT DEFAULT 'team' NOT NULL CHECK(role IN ('owner', 'team', 'readonly')), joined_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); ``` | Column | Type | Description | |--------|------|-------------| | `id` | INTEGER | Primary key | | `player_name` | TEXT | Minecraft username (unique) | | `role` | TEXT | 'owner', 'team', or 'readonly' | | `joined_at` | TIMESTAMP | When player was added | #### `chests` Tracked chest blocks in storage area. ```sql CREATE TABLE chests ( id INTEGER PRIMARY KEY AUTOINCREMENT, pos_x INTEGER NOT NULL, pos_y INTEGER NOT NULL, pos_z INTEGER NOT NULL, chest_type TEXT NOT NULL CHECK(chest_type IN ('single', 'double')), row INTEGER NOT NULL, -- 1-4 (vertical) column INTEGER NOT NULL, -- horizontal grouping category TEXT, -- 'minerals', 'food', etc. last_scan TIMESTAMP DEFAULT CURRENT_TIMESTAMP, UNIQUE(pos_x, pos_y, pos_z) ); ``` #### `shulkers` Shulker boxes stored in chests. ```sql CREATE TABLE shulkers ( id INTEGER PRIMARY KEY AUTOINCREMENT, chest_id INTEGER NOT NULL, slot INTEGER NOT NULL, -- 0-53 (single) or 0-107 (double) shulker_type TEXT DEFAULT 'shulker_box', category TEXT, -- 'minerals', 'tools', etc. item_focus TEXT, -- Item type stored (e.g., 'diamond') slot_count INTEGER DEFAULT 27, -- Used slots (1-27) total_items INTEGER DEFAULT 0, -- Total item count last_scan TIMESTAMP DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (chest_id) REFERENCES chests(id) ON DELETE CASCADE ); ``` #### `shulker_items` Items inside shulker boxes. Enforces one item type per shulker. ```sql CREATE TABLE shulker_items ( id INTEGER PRIMARY KEY AUTOINCREMENT, shulker_id INTEGER NOT NULL, item_name TEXT NOT NULL, item_id INTEGER NOT NULL, slot INTEGER NOT NULL, -- 0-26 (shulker slots) count INTEGER NOT NULL, nbt_data TEXT, -- JSON: {enchantments: [...], damage: 5} FOREIGN KEY (shulker_id) REFERENCES shulkers(id) ON DELETE CASCADE, UNIQUE(shulker_id, item_id), CHECK(slot >= 0 AND slot <= 26), CHECK(count > 0 AND count <= 64) ); ``` #### `trades` Trade history logs. ```sql CREATE TABLE trades ( id INTEGER PRIMARY KEY AUTOINCREMENT, player_name TEXT NOT NULL, action TEXT NOT NULL CHECK(action IN ('deposit', 'withdraw')), items TEXT NOT NULL, -- JSON: [{name, count, nbt}, ...] timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); ``` #### `pending_withdrawals` Withdrawal requests from players (sync between web and in-game). ```sql CREATE TABLE pending_withdrawals ( id INTEGER PRIMARY KEY AUTOINCREMENT, player_name TEXT NOT NULL, item_id INTEGER NOT NULL, item_name TEXT NOT NULL, requested_count INTEGER NOT NULL, status TEXT DEFAULT 'pending' CHECK(status IN ('pending', 'ready', 'completed', 'cancelled')), timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); ``` #### `item_index` Cached aggregated item counts for fast searches. ```sql CREATE TABLE item_index ( id INTEGER PRIMARY KEY AUTOINCREMENT, item_id INTEGER UNIQUE NOT NULL, item_name TEXT NOT NULL, total_count INTEGER DEFAULT 0, shulker_ids TEXT, -- JSON: [{id, count}, ...] last_updated TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); ``` --- ## File Structure ``` nodejs/ ├── controller/ │ ├── storage/ │ │ ├── index.js # StoragePlugin main class │ │ ├── database.js # SQLite setup and all DB operations │ │ ├── scanner.js # Chest discovery and scanning │ │ ├── organizer.js # Item sorting and categorization │ │ └── web.js # Express server (24/7 API) │ ├── storage.js # Export for mc-bot.js plugin loading │ └── commands/ │ ├── default.js # Add: summon, dismiss commands │ └── trade.js # Add StoragePlugin special handling ├── storage/ │ ├── storage.db # SQLite database (automatically created) │ └── public/ │ ├── index.html # Web UI (single page) │ ├── app.js # Frontend logic │ └── style.css # Styling ├── conf/ │ ├── base.js # Add storage config │ └── secrets.js # DB path, permissions init └── ... ``` --- ## Component Specifications ### 1. StoragePlugin (`controller/storage/index.js`) **Purpose**: Main plugin class that ties together database, scanner, organizer, and trade handling. **Constructor Arguments**: - `bot`: The CJbot instance - `dbFile`: Path to SQLite database - `homePos`: Starting position (optional, auto-detect on first scan) **Key Methods**: ```javascript class StoragePlugin { constructor(args) { ... } async init() { // Initialize database // Register commands // Start on bot 'onReady' } async unload() { // Clean up } async scanArea(force = false) { // Discover chests within render distance // Update database } async handleTrade(playerName, itemsReceived) { // Process incoming trade items // Sort into shulkers // Update database } async handleWithdrawRequest(playerName, itemId, count) { // Gather items to OUTBOX shulker // Mark as ready for pickup } async organize() { // Full re-sort (manual command) } } ``` ### 2. Database Module (`controller/storage/database.js`) **Purpose**: All SQLite operations. **Key Functions**: ```javascript // Initialize async initialize(dbFile) // Permissions async addPlayer(name, role = 'team') async removePlayer(name) async getPlayerRole(name) async getAllPlayers() async checkPermission(name, requiredRole) // Chests async upsertChest(position, chestType) async getChests() async deleteOrphanChests() // Shulkers async upsertShulker(chestId, slot, category, itemFocus) async getShulkersByChest(chestId) async findShulkerForItem(itemId) // Find shulker with same item and space async createEmptyShulker(chestId, slot) async updateShulkerCounts(shulkerId, slotCount, totalItems) // Shulker Items async upsertShulkerItem(shulkerId, item) async getShulkerItems(shulkerId) async deleteShulkerItem(shulkerId, itemId) // Trades async logTrade(playerName, action, items) async getRecentTrades(limit = 50) // Pending Withdrawals async queueWithdrawal(playerName, itemId, itemName, count) async getPendingWithdrawals(playerName) async updateWithdrawStatus(id, status) async markCompletedWithdrawals(playerName) // Item Index async updateItemCount(itemId, shulkerId, count) async searchItems(query = null) async getItemDetails(itemId) ``` ### 3. Scanner Module (`controller/storage/scanner.js`) **Purpose**: Discover and scan chests/shulkers. **Key Functions**: ```javascript async discoverChests(bot, radius) { // Find all chest blocks within radius // Detect single vs double // Assign row/column based on position // Return array of chest positions } async scanChest(bot, chestPosition) { // Open chest // Read all slots // Scan any shulkers found // Update database } async scanShulker(bot, chestSlot, chestPosition) { // Click shulker to open // Read all 27 slots // Parse NBT data // Return item array } function detectChestType(position) { // Check adjacent blocks to detect double chest // Return 'single' or 'double' } function assignRowColumn(position, minPos) { // Calculate row from Y (1-4) // Calculate column from X/Z // Return {row, column} } ``` ### 4. Organizer Module (`controller/storage/organizer.js`) **Purpose**: Sort items into shulkers, categorize items. **Key Functions**: ```javascript function categorizeItem(itemName) { // Returns: 'minerals', 'food', 'tools', 'armor', 'blocks', 'redstone', 'misc' } async sortItems(itemsDb, itemsToSort) { // For each item: // - Find shulker with same item AND space // - If found: move to that shulker, consolidate stacks // - If not found: create new shulker at category column // Return: moves to execute } function findCategoryColumn(category, row) { // Map (category, row) to chest column // Return column number } async consolidateStacks(shulkerId) { // Merge partial stacks // Update database } ``` ### 5. Web Server (`controller/storage/web.js`) **Purpose**: Express API for 24/7 inventory access. **API Endpoints**: ``` GET /api/inventory GET /api/inventory/:itemId GET /api/chests GET /api/chests/:id GET /api/shulkers GET /api/shulkers/:id GET /api/stats GET /api/trades?limit=50 POST /api/withdraw GET /api/pending/:playerName POST /api/auth ``` **Detailed API Spec**: ``` GET /api/inventory Response: { items: [ { item_id: 1, item_name: "diamond", total_count: 2304, locations: [ {shulker_id: 1, count: 1728}, {shulker_id: 5, count: 576} ] }, ... ] } GET /api/inventory/:itemId Response: { item_id: 1, item_name: "diamond", total_count: 2304, locations: [ { shulker_id: 1, chest_id: 3, chest_pos: {x: 100, y: 64, z: 200}, count: 1728 }, ... ] } GET /api/chests Response: { chests: [ { id: 1, pos_x: 100, pos_y: 64, pos_z: 200, chest_type: "double", row: 1, column: 1, category: "minerals", shulker_count: 26 }, ... ] } GET /api/stats Response: { totalItems: 15432, totalShulkers: 156, totalChests: 24, emptyShulkers: 12, categories: { minerals: 42, food: 18, tools: 24, ... }, recentTrades: [ {player: "wmantly", action: "deposit", item_count: 45, time: "..."} ] } POST /api/withdraw Body: {player_name: "wmantly", item_id: 1, count: 64} Response: {success: true, withdraw_id: 123} GET /api/pending/:playerName Response: { pending: [ { id: 123, item_name: "diamond", requested_count: 64, status: "ready" } ] } ``` ### 6. Web UI (`storage/public/`) **index.html**: Single page application - Search bar - Filter by category - Item list with counts - Click to see details (shulker locations) - "Request Withdraw" button (opens modal) - Stats sidebar **app.js**: Frontend logic - Fetch API calls - Search/filter logic - Withdraw request modal - Auto-refresh pending withdrawals **style.css**: Simple, clean styling ### 7. Modified Commands **default.js** - Add new commands: ```javascript 'summon': { desc: 'Summon a bot online indefinitely', allowed: ['owner'], ignoreLock: true, async function(from, botName) { ... } }, 'dismiss': { desc: 'Send a bot offline', allowed: ['owner'], ignoreLock: true, async function(from, botName) { ... } }, ``` **trade.js** - Add StoragePlugin handling: ```javascript module.exports = { '.trade': { desc: 'Bot will take trade requests', async function(from) { // Check if bot has StoragePlugin if (this.plunginsLoaded['Storage']) { await this.plunginsLoaded['Storage'].handleTrade(from, ...); } else { // Original sign-based flow let chestBlock = findChestBySign(this, from); // ... } } } } ``` --- ## Configuration ### conf/base.js ```javascript "storage": { // Database location "dbPath": "./storage/storage.db", // Chest discovery "scanRadius": 30, // Render distance "homePos": null, // Auto-detect on first scan // Category mappings "categories": { "minerals": ["diamond", "netherite_ingot", "gold_ingot", "iron_ingot", "copper_ingot", "emerald", "redstone", "lapis_lazuli"], "food": ["bread", "cooked_porkchop", "steak", "golden_apple", "cooked_beef", "cooked_chicken", "cooked_mutton", "carrot", "potato", "baked_potato"], "tools": ["wooden_sword", "stone_sword", "iron_sword", "diamond_sword", "netherite_sword", "wooden_pickaxe", "stone_pickaxe", "iron_pickaxe", "diamond_pickaxe", "netherite_pickaxe", "wooden_axe", "stone_axe", "iron_axe", "diamond_axe", "netherite_axe"], "armor": ["leather_helmet", "iron_helmet", "diamond_helmet", "netherite_helmet", "leather_chestplate", "iron_chestplate", "diamond_chestplate", "netherite_chestplate", "leather_leggings", "iron_leggings", "diamond_leggings", "netherite_leggings", "leather_boots", "iron_boots", "diamond_boots", "netherite_boots"], "blocks": ["stone", "dirt", "cobblestone", "oak_planks", "spruce_planks", "birch_planks", "oak_log", "spruce_log", "cobblestone_stairs"], "redstone": ["redstone", "repeater", "comparator", "piston", "sticky_piston", "redstone_torch", "lever", "tripwire_hook"], "misc": [] // Everything else falls here }, // Special shulkers (for bookkeeping) "inboxShulkerName": "INBOX", "outboxShulkerName": "OUTBOX", "newShulkersName": "EMPTY", // Web server "webPort": 3000, "webHost": "0.0.0.0" } ``` ### conf/secrets.js ```javascript "storage": { // Database can override location // "dbPath": "./storage/storage.db", // Default permissions (inserted on DB init) "defaultPlayers": [ {name: "wmantly", role: "owner"}, {name: "useless666", role: "owner"}, {name: "tux4242", role: "owner"}, {name: "pi_chef", role: "team"}, {name: "Ethan", role: "team"}, {name: "Vince_NL", role: "team"} ] } ``` ### Bot Configuration (Plugin-Based, Bot-Agnostic) ```javascript // conf/secrets.js - Example: ez bot "mc": { "bots": { "ez": { "username": "mc3@vm42.us", "auth": "microsoft", "commands": ['default'], "autoConnect": false, "plugins": { "Storage": { // Bot uses Storage plugin } } }, // Any bot can be moved to storage location: "art": { "username": "art@vm42.us", "auth": "microsoft", "plugins": { "Storage": { // Same plugin, different bot } } } } } ``` --- ## User Workflows ### Deposit Flow (Player Perspective) 1. **Player collects items** from farms/raids (max 12 slots due to trade window) 2. **Player types**: `/msg ez trade` or uses `/trade` command with ez 3. **Trade window opens** 4. **Player puts items** in their side of trade window 5. **Confirm trade** 6. **ez automatically**: - Moves all items to INBOX shulker - Categorizes each item - Finds appropriate shulker (creates new if needed) - Moves items to organized shulkers - Updates database 7. **ez whispers**: `Received X items. Stored successfully.` 8. **Player can verify** on web: `http://server:3000` ### Withdraw Flow (Player Perspective) **Option A: In-Game Only** 1. **Player types**: `/msg ez withdraw diamond 64` 2. **ez searches database** for diamond locations 3. **ez gathers items** to OUTBOX shulker 4. **ez whispers**: `Items ready for pickup. /trade with me.` 5. **Player trades** with ez 6. **ez moves items** from OUTBOX to trade window 7. **Confirm trade** 8. **ez updates database** **Option B: Web Request** 1. **Player visits** web: `http://server:3000` 2. **Finds item** and enters count 3. **Click "Withdraw"** 4. **Queues request** to database 5. **When ez is online**, processes pending requests 6. **ez whispers player**: `Your items are ready. /trade with me.` ### Admin Workflow (Owner) ``` /msg ez summon # Bring bot online /msg ez dismiss # Send bot offline /msg ez scan # Force chest scan /msg ez chests # List tracked chests /msg ez organize # Force full re-sort /msg ez status # Show storage stats /msg ez addplayer # Add authorized player /msg ez removeplayer # Remove player /msg ez players # List authorized players ``` --- ## Security Considerations ### Database Security - SQLite file permissions: Read/write by bot process only - No direct SQL injection (parameterized queries throughout) - Web API uses read-only connections for GET requests ### Game Security - Bot location kept secret (no `/tp` to chests allowed to team) - Only trade window access for players - No `/msg` command execution to other bots ### Web Security - No login required (simpler) - Auth via whisper challenge code (6-digit code generated, whispered to player for verification) - Rate limiting on API endpoints - CORS restricted to same origin --- ## Implementation Plan ### Phase 1: Core Database and Plugin Structure - Create `database.js` - SQLite setup and all queries - Create `index.js` - StoragePlugin main class skeleton - Initialize plugin in `mc-bot.js` via plugin system - Test: Database creation, basic connectivity ### Phase 2: Scanner - Create `scanner.js` - Chest discovery and scanning - Scan loop: Find chests → Detect single/double → Assign row/column - Scan individual chests → Detect shulkers → Read NBT → Update DB - Test: Scan a test chest area, verify DB entries ### Phase 3: Organizer - Create `organizer.js` - Categorization and sorting - Implement `categorizeItem()` function - Implement sort logic (find shulker, move items, create new if needed) - Test: Sort items from INBOX to organized shulkers ### Phase 4: Trade Integration - Modify `trade.js` - Add StoragePlugin handling - Implement `handleTrade()` in StoragePlugin - Implement `handleWithdrawRequest()` in StoragePlugin - Test: Deposit and withdraw via trade window ### Phase 5: Commands - Add `summon`, `dismiss` to `default.js` - Add storage commands (`scan`, `status`, `chests`, `organize`) - Add player management commands (`addplayer`, `removeplayer`, `players`) - Test: All commands work with proper permissions ### Phase 6: Web Server - Create `web.js` - Express server - Implement API endpoints - Test: API returns correct data from database ### Phase 7: Web UI - Create `index.html` - Single page UI - Create `app.js` - Frontend logic - Create `style.css` - Styling - Test: View inventory, search, request withdrawal ### Phase 8: Integration Testing - Full deposit flow end-to-end - Full withdraw flow end-to-end - Web + in-game sync - Multiple trades in sequence - Database persistence after bot restart ### Phase 9: Documentation and Polish - Update this doc with any changes - Add inline code comments - Error handling improvements - Performance optimization if needed --- ## Dependencies ### New npm packages required: ``` sqlite3 # SQLite database express # Web server cors # CORS handling (optional, for external API access) ``` Add to `package.json`: ```json { "dependencies": { "sqlite3": "^5.1.7", "express": "^4.19.2", "cors": "^2.8.5" } } ``` --- ## Troubleshooting Guide ### Database Issues - **Database locked**: Ensure only one process writes at a time - **Corruption**: Use `sqlite3 storage.db "PRAGMA integrity_check;"` to verify ### Scan Issues - **No chests found**: Check `homePos` is correct, or bot is in render distance - **Double chest detection fails**: Ensure chests are properly adjacent ### Trade Issues - **Items not sorted**: Check INBOX shulker exists and has space - **Withdraw fails**: Verify item exists in database with sufficient count ### Web Issues - **Can't access API**: Check `webHost` (use `0.0.0.0` for external access) - **Database not found**: Ensure `dbPath` directory exists and has write permissions --- ## Future Enhancements (Out of Scope for MVP) - [ ] Shulker color coding for categories (needs dye access, blocked by server rules) - [ ] Custom shulker names via NBT (blocked by server rules) - [ ] Partial shulker consolidation (current: one item type per shulker, no combining) - [ ] Auto-trading system (bot initiates trades) - [ ] Multi-location support (multiple storage areas) - [ ] Real-time web updates via WebSocket - [ ] Export inventory to CSV/JSON - [ ] Integration with trading APIs (like BulbaStore) - [ ] Recipe calculator (what can be crafted with current storage) - [ ] Value estimation (diamond equivalent of all items) --- ## Glossary | Term | Meaning | |------|---------| | **StoragePlugin** | The main plugin class that provides storage functionality to any bot | | **INBOX shulker** | Temporary holding shulker for incoming trade items | | **OUTBOX shulker** | Temporary holding shulker for items ready for withdrawal | | **EMPTY shulkers** | Reserve stock of new shulker boxes | | **One item type per shulker** | Each shulker stores only one item type (e.g., only diamonds) | | **Category** | Item group (minerals, food, tools, armor, blocks, redstone, misc) | | **Row/Column** | Chest positioning: Row (1-4, vertical), Column (horizontal grouping) | | **Render distance** | Distance within which bot can see/click chests |