From 354abdf19b8466a2c681354135a3a584d30d2b46 Mon Sep 17 00:00:00 2001 From: William Mantly Date: Wed, 31 Dec 2025 16:51:27 -0500 Subject: [PATCH] Gitpage added --- docs/README.md | 44 ++++ docs/_config.yml | 9 + docs/api.md | 549 +++++++++++++++++++++++++++++++++++++++++++ docs/architecture.md | 265 +++++++++++++++++++++ docs/contributing.md | 308 ++++++++++++++++++++++++ docs/index.md | 97 ++++++++ docs/installation.md | 221 +++++++++++++++++ 7 files changed, 1493 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/_config.yml create mode 100755 docs/api.md create mode 100644 docs/architecture.md create mode 100644 docs/contributing.md create mode 100644 docs/index.md create mode 100644 docs/installation.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..e23feb5 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,44 @@ +# Documentation + +This directory contains the GitHub Pages documentation site for the Proxy project. + +**Live site:** https://theta42.github.io/proxy/ + +## Pages + +- `index.md` - Home page with project overview +- `installation.md` - Installation and setup guide +- `api.md` - Complete API reference +- `architecture.md` - System architecture and design +- `contributing.md` - Development and contribution guide + +## Local Preview + +To preview the site locally: + +```bash +# Install Jekyll (one-time setup) +gem install jekyll bundler + +# Run local server +cd docs +jekyll serve + +# View at http://localhost:4000/proxy/ +``` + +## Theme + +The site uses the Cayman theme (`jekyll-theme-cayman`). Configuration is in `_config.yml`. + +## Updating Documentation + +1. Edit markdown files in this directory +2. Commit and push to master branch +3. GitHub Pages automatically rebuilds (may take 1-2 minutes) +4. Changes visible at https://theta42.github.io/proxy/ + +## Legacy Documentation + +- `dev_setup.md` - Old development setup notes (kept for reference) +- `Update 4.11.md` - Old update notes (kept for reference) diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 0000000..71c46c3 --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1,9 @@ +title: Proxy +description: A reverse proxy and HTTPS termination service using OpenResty/nginx with a management API and web GUI +theme: jekyll-theme-cayman +show_downloads: true +github: + repository_url: https://github.com/theta42/proxy + zip_url: https://github.com/theta42/proxy/archive/refs/heads/master.zip + tar_url: https://github.com/theta42/proxy/archive/refs/heads/master.tar.gz + repository_name: theta42/proxy diff --git a/docs/api.md b/docs/api.md new file mode 100755 index 0000000..9ce0536 --- /dev/null +++ b/docs/api.md @@ -0,0 +1,549 @@ +--- +layout: default +title: API Reference +--- + +# API Documentation + +[← Back to Home](index.html) + +All API endpoints require authentication via the `auth-token` header unless otherwise noted. + +Base URL: `https://your-proxy-host.com/api` + +--- + +## Authentication + +### Login + +**POST** `/api/auth/login` + +Authenticate a user and receive an auth token. + +```bash +curl -H "Content-Type: application/json" \ + -X POST \ + -d '{"username": "myuser", "password": "mypassword"}' \ + https://proxy-host.com/api/auth/login +``` + +**Responses:** +- `200` `{"login": true, "token": "027d3964-7d81-4462-a6f9-2c1f9b40b4be", "message": "myuser logged in!"}` +- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` + +### Logout + +**ALL** `/api/auth/logout` + +Invalidate the current auth token. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/auth/logout +``` + +**Responses:** +- `200` `{"message": "Bye"}` + +--- + +## Users + +All user endpoints require authentication. + +### List Users + +**GET** `/api/user` + +Get list of all users. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/user +``` + +**Query Parameters:** +- `detail` - Include full user details (optional) + +**Responses:** +- `200` `{"results": ["user1", "user2"]}` +- `200` `{"results": [{"username": "user1", ...}, ...]}` (with `?detail=true`) + +### Get Current User + +**GET** `/api/user/me` + +Get information about the currently authenticated user. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/user/me +``` + +**Responses:** +- `200` `{"username": "myuser"}` + +### Create User + +**POST** `/api/user` + +Create a new user. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"username": "newuser", "password": "newpassword"}' \ + https://proxy-host.com/api/user +``` + +**Responses:** +- `200` User created successfully +- `409` Username already exists +- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error + +### Delete User + +**DELETE** `/api/user/:username` + +Delete a user account. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/user/olduser +``` + +**Responses:** +- `200` `{"username": "olduser", "results": ...}` +- `404` User not found + +### Change Password (Self) + +**PUT** `/api/user/password` + +Change the password for the currently authenticated user. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"password": "newpassword"}' \ + https://proxy-host.com/api/user/password +``` + +**Responses:** +- `200` `{"results": ...}` Password changed successfully + +### Change Password (Other User) + +**PUT** `/api/user/password/:username` + +Change the password for another user (admin function). + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"password": "newpassword"}' \ + https://proxy-host.com/api/user/password/otheruser +``` + +**Responses:** +- `200` `{"results": ...}` Password changed successfully +- `404` User not found + +### Create Invite Token + +**POST** `/api/user/invite` + +Create an invitation token for new user registration. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/user/invite +``` + +**Responses:** +- `200` `{"token": "5caf94d2-2c91-4010-8df7-968d10802b9d"}` + +### Add SSH Key + +**POST** `/api/user/key` + +Add an SSH public key to the current user's account. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"key": "ssh-rsa AAAAB3..."}' \ + https://proxy-host.com/api/user/key +``` + +**Responses:** +- `200` `{"message": true}` Key added successfully +- `400` `{"message": "Bad SSH key"}` Invalid key format + +--- + +## Hosts + +Manage proxy host configurations. + +### List Hosts + +**GET** `/api/host` + +Get list of all configured hosts. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host +``` + +**Query Parameters:** +- `detail` - Include full host details (optional) + +**Responses:** +- `200` `{"results": ["example.com", "*.wildcard.com"]}` +- `200` `{"results": [{"host": "example.com", "ip": "192.168.1.10", ...}, ...]}` (with `?detail=true`) + +### Get Host + +**GET** `/api/host/:host` + +Get configuration for a specific host. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"item": "example.com", "results": {"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, ...}}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` + +### Lookup Host + +**GET** `/api/host/lookup/:domain` + +Test the host lookup algorithm (supports wildcard matching). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/lookup/sub.example.com +``` + +**Responses:** +- `200` `{"string": "sub.example.com", "results": {"host": "*.example.com", ...}}` +- `200` `{"string": "sub.example.com", "results": null}` (no match) + +### Get Lookup Tree + +**GET** `/api/host/lookupobj` + +Get the internal lookup tree structure (for debugging). + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/host/lookupobj +``` + +**Responses:** +- `200` `{"results": {"com": {"example": {...}}}}` + +### Create Host + +**POST** `/api/host` + +Add a new host configuration. + +**Parameters:** +- `host` (required) - Domain name (e.g., `example.com`, `*.example.com`) +- `ip` (required) - Target IP address or FQDN +- `targetPort` (required) - Target port number (1-65535) +- `forcessl` (optional) - Force HTTPS redirect (default: true) +- `targetssl` (optional) - Use HTTPS to backend (default: false) +- `challengeType` (optional) - For wildcards: `DNS-01-wildcard` or `wildcardChild` + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, "forcessl": true, "targetssl": false}' \ + https://proxy-host.com/api/host +``` + +**Responses:** +- `200` `{"message": "\"example.com\" added.", "host": "example.com", ...}` +- `409` `{"name": "HostNameUsed", "message": "Host already exists"}` +- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error + +### Update Host + +**PUT** `/api/host/:host` + +Update an existing host configuration. + +**Parameters:** Same as Create Host (all optional) + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"ip": "192.168.1.20", "targetPort": 9000}' \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"message": "\"example.com\" updated.", ...}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` +- `422` Validation error + +### Delete Host + +**DELETE** `/api/host/:host` + +Remove a host configuration. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/host/example.com +``` + +**Responses:** +- `200` `{"message": "example.com deleted", ...}` +- `404` `{"name": "HostNotFound", "message": "Host does not exists"}` + +### Renew Wildcard Certificate + +**PUT** `/api/host/:host/renew` + +Manually trigger wildcard certificate renewal. + +```bash +curl -H "auth-token: your-token-here" \ + -X PUT \ + https://proxy-host.com/api/host/*.example.com/renew +``` + +**Responses:** +- `200` `{"message": "Requesting wildcard cert for *.example.com"}` +- `404` Host not found + +--- + +## DNS Providers + +Manage DNS provider integrations for wildcard SSL certificates. + +### List DNS Providers + +**GET** `/api/dns` + +Get list of configured DNS providers. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns +``` + +**Query Parameters:** +- `detail` - Include full provider details (optional) + +**Responses:** +- `200` `{"results": ["provider-id-1", "provider-id-2"]}` + +### List Available Provider Types + +**OPTIONS** `/api/dns` + +Get list of supported DNS provider types and their configuration requirements. + +```bash +curl -H "auth-token: your-token-here" \ + -X OPTIONS \ + https://proxy-host.com/api/dns +``` + +**Responses:** +- `200` `{"results": [{"name": "CloudFlare", "fields": {...}}, {"name": "DigitalOcean", ...}, ...]}` + +### Create DNS Provider + +**POST** `/api/dns` + +Configure a new DNS provider. + +**CloudFlare:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My CloudFlare", "dnsProvider": "Cloudflare", "token": "your-api-token"}' \ + https://proxy-host.com/api/dns +``` + +**DigitalOcean:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My DO", "dnsProvider": "DigitalOcean", "token": "your-api-token"}' \ + https://proxy-host.com/api/dns +``` + +**PorkBun:** +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X POST \ + -d '{"name": "My PorkBun", "dnsProvider": "PorkBun", "apiKey": "pk_xxx", "secretApiKey": "sk_xxx"}' \ + https://proxy-host.com/api/dns +``` + +**Responses:** +- `200` `{"message": "\"provider-id\" added.", ...}` +- `422` Validation error or invalid API credentials + +### Get DNS Provider + +**GET** `/api/dns/:id` + +Get a specific DNS provider configuration. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"item": "provider-id", "results": {...}}` +- `404` Provider not found + +### Update DNS Provider + +**PUT** `/api/dns/:id` + +Update DNS provider configuration. + +```bash +curl -H "Content-Type: application/json" \ + -H "auth-token: your-token-here" \ + -X PUT \ + -d '{"name": "Updated Name"}' \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"message": "\"provider-id\" updated.", ...}` +- `404` Provider not found + +### Delete DNS Provider + +**DELETE** `/api/dns/:id` + +Remove a DNS provider and all associated domains. + +```bash +curl -H "auth-token: your-token-here" \ + -X DELETE \ + https://proxy-host.com/api/dns/provider-id +``` + +**Responses:** +- `200` `{"message": "provider-id deleted", ...}` +- `404` Provider not found + +### List Domains + +**GET** `/api/dns/domain` + +List all domains from all configured providers. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/domain +``` + +**Query Parameters:** +- `detail` - Include full domain details (optional) + +**Responses:** +- `200` `{"results": ["example.com", "test.com"]}` + +### Get Domain + +**GET** `/api/dns/domain/:domain` + +Get details for a specific domain. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/dns/domain/example.com +``` + +**Responses:** +- `200` `{"results": [{"domain": "example.com", "zoneId": "...", ...}]}` +- `404` Domain not found + +### Refresh Domains + +**POST** `/api/dns/domain/refresh/:providerId` + +Refresh the domain list from a DNS provider's API. + +```bash +curl -H "auth-token: your-token-here" \ + -X POST \ + https://proxy-host.com/api/dns/domain/refresh/provider-id +``` + +**Responses:** +- `200` `{"results": ...}` Updated domain list +- `404` Provider not found + +--- + +## Certificates + +Retrieve SSL certificate information. + +### Get Certificate + +**GET** `/api/cert/:host` + +Get the SSL certificate for a host. + +```bash +curl -H "auth-token: your-token-here" \ + https://proxy-host.com/api/cert/example.com +``` + +**Responses:** +- `200` Certificate data including `cert_pem`, `fullchain_pem`, `privkey_pem`, expiry information +- `404` Certificate not found + +--- + +## Error Responses + +All endpoints may return the following error responses: + +- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - Authentication required or invalid +- `404` `{"name": "NotFound", "message": "..."}` - Resource not found +- `422` `{"name": "ObjectValidateError", "message": [...], "keys": [...]}` - Validation errors +- `500` Internal server error + +## Notes + +- All timestamps are in milliseconds since epoch +- The `auth-token` header is required for all authenticated endpoints +- Host names support wildcards: `*` (single level) and `**` (multi-level) +- DNS providers are validated on creation - invalid API credentials will be rejected +- Wildcard certificates are automatically renewed 30 days before expiration diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..36f18de --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,265 @@ +--- +layout: default +title: Architecture +--- + +# Architecture + +[← Back to Home](index.html) + +## System Overview + +The proxy system consists of three main components working together to provide high-performance reverse proxying with automated SSL management. + +``` +┌──────────────────────────────────────────────────────────────┐ +│ Internet │ +└─────────────────────────┬────────────────────────────────────┘ + │ HTTPS/HTTP + ▼ +┌──────────────────────────────────────────────────────────────┐ +│ OpenResty/Nginx │ +│ ┌────────────────┐ ┌──────────────┐ ┌─────────────────┐ │ +│ │ SSL Termination│ │ Host Routing │ │ Request Proxying│ │ +│ │ (lua-resty- │ │ (targetinfo. │ │ │ │ +│ │ auto-ssl) │ │ lua) │ │ │ │ +│ └────────────────┘ └──────────────┘ └─────────────────┘ │ +└────────────┬──────────────────┬───────────────────────────┬──┘ + │ │ │ + Let's Encrypt Unix Socket Backend + HTTP-01 Lookup Query Services + │ │ │ + ▼ ▼ ▼ +┌──────────────────────────────────────────────────────────────┐ +│ Node.js Application │ +│ ┌──────────────┐ ┌────────────────┐ ┌─────────────────┐ │ +│ │ Services │ │ Models │ │ Routes │ │ +│ │ - host_lookup│ │ - Host │ │ - /api/host │ │ +│ │ - scheduler │ │ - DNS Provider │ │ - /api/dns │ │ +│ └──────────────┘ │ - User │ │ - /api/user │ │ +│ │ - Auth │ │ - /api/auth │ │ +│ └────────────────┘ │ - /api/cert │ │ +│ └─────────────────┘ │ +└────────────┬────────────────────┬────────────────────────────┘ + │ │ + ▼ ▼ +┌──────────────────────┐ ┌──────────────────────┐ +│ Redis │ │ DNS Providers │ +│ - Host configs │ │ - CloudFlare │ +│ - User accounts │ │ - DigitalOcean │ +│ - SSL certs │ │ - PorkBun │ +│ - Auth tokens │ │ (DNS-01 challenges) │ +└──────────────────────┘ └──────────────────────┘ +``` + +## Component Details + +### OpenResty/Nginx (Frontend) + +**Responsibilities:** +- Accept incoming HTTP/HTTPS requests +- SSL termination using lua-resty-auto-ssl +- Host-based routing decisions +- Proxy requests to backend services + +**Key Features:** +- HTTP-01 ACME challenge handling for automatic SSL +- Lua-based host lookup via Unix socket +- High-performance event-driven architecture +- Support for WebSocket connections + +**Configuration Files:** +- `/etc/openresty/nginx.conf` - Main configuration +- `/etc/openresty/autossl.conf` - Let's Encrypt integration +- `/etc/openresty/sites-enabled/000-proxy` - Proxy configuration +- `/usr/local/openresty/lualib/targetinfo.lua` - Host lookup module + +### Node.js Application (Backend) + +**Responsibilities:** +- API for host/user/DNS management +- Wildcard SSL certificate orchestration +- Host lookup tree maintenance +- User authentication and authorization + +**Directory Structure:** +``` +nodejs/ +├── bin/www # Application entry point +├── models/ # Data models +│ ├── host.js # Host configuration and lookup +│ ├── auth.js # Authentication logic +│ ├── user.js # User management +│ └── dns_provider/ # DNS provider implementations +├── routes/ # API endpoints +│ ├── host.js # Host CRUD operations +│ ├── dns.js # DNS provider management +│ ├── user.js # User management +│ └── auth.js # Authentication +├── services/ # Background services +│ ├── host_lookup.js # Unix socket server +│ └── host_scheduler.js # Cert renewal scheduler +├── middleware/ # Express middleware +│ └── auth.js # Authentication middleware +└── utils/ # Utility modules + └── unix_socket_json.js # Unix socket server +``` + +### Redis (Data Store) + +**Stored Data:** +- Host configurations (domain, IP, port, SSL settings) +- User accounts and hashed passwords +- Authentication tokens +- SSL certificates (for wildcard domains) +- DNS provider credentials +- Domain-to-provider mappings + +**Key Prefixes:** +``` +proxy_Host_ # Host configuration +proxy_User_ # User account +proxy_AuthToken_ # Auth tokens +proxy_DnsProvider_ # DNS provider +proxy_Domain_ # Domain info +:latest # SSL certificate cache +``` + +## Request Flow + +### Standard HTTP/HTTPS Request + +1. **Client** sends HTTPS request to `app.example.com` +2. **OpenResty** receives request, terminates SSL +3. **Lua script** (`targetinfo.lua`) queries Redis for host config +4. If **cache miss**, Lua queries Node.js via Unix socket +5. **Node.js** performs host lookup (supports wildcards) +6. **Response** returned with target IP and port +7. **OpenResty** proxies request to backend service +8. **Response** proxied back to client + +### Wildcard SSL Certificate Request + +1. **User** creates wildcard host (`*.example.com`) via API +2. **Node.js** validates domain has DNS provider configured +3. **Let's Encrypt** DNS-01 challenge initiated +4. **DNS provider** API creates TXT record (`_acme-challenge.example.com`) +5. **Let's Encrypt** validates TXT record +6. **Certificate** generated and stored in Redis +7. **DNS provider** cleans up TXT record +8. **Background scheduler** monitors expiration, renews 30 days before expiry + +## Host Lookup Algorithm + +The lookup tree enables sophisticated domain matching: + +``` +Input: "api.v1.example.com" + +Tree Structure: +{ + "com": { + "example": { + "*": { // Matches api.example.com + "#record": {...} + }, + "v1": { + "api": { // Matches api.v1.example.com (exact) + "#record": {...} + } + } + } + } +} + +Priority: Exact > Single wildcard (*) > Double wildcard (**) +``` + +**Wildcard Types:** +- `example.com` - Exact match only +- `*.example.com` - Matches `sub.example.com` (single level) +- `**.example.com` - Matches any depth (`sub.deep.example.com`) +- `api.*.example.com` - Matches `api.v1.example.com`, `api.v2.example.com` + +## Security Architecture + +### Authentication Flow + +1. User sends credentials to `/api/auth/login` +2. Credentials validated against stored hash (bcrypt) +3. Token generated and stored in Redis with TTL +4. Token returned to client +5. Subsequent requests include token in `auth-token` header +6. Middleware validates token before processing request + +### SSL Certificate Security + +- **Private keys** stored only in Redis (memory/disk based on config) +- **Fallback certificates** used when SNI unavailable +- **Let's Encrypt** rate limiting respected +- **DNS provider credentials** marked as `isPrivate` (not returned in API) + +### Unix Socket Communication + +- Socket file: `/var/run/proxy_lookup.socket` +- Permissions: `777` (container-safe, single-use deployment) +- Protocol: JSON over Unix stream socket +- Buffer handling: Accumulates partial messages until complete JSON + +## Performance Optimizations + +### Caching Strategy + +1. **Redis cache** - Primary host configuration storage +2. **Lookup tree** - In-memory host lookup (rebuilt on changes) +3. **OpenResty cache** - Reduces Unix socket calls +4. **Wildcard parent caching** - Stores resolved wildcard parents + +### Unix Socket vs HTTP API + +Unix socket chosen over HTTP for host lookups: +- **Lower latency** - No TCP overhead +- **Higher throughput** - No HTTP parsing +- **Simpler** - Direct JSON communication +- **Secure** - Filesystem permissions, no network exposure + +## Scalability Considerations + +### Current Architecture + +- **Single instance** - OpenResty + Node.js + Redis on one server +- **Vertical scaling** - Add CPU/RAM as needed +- **Limitations** - Unix socket ties OpenResty to Node.js on same host + +### Future Scaling Options + +- **Redis cluster** - Distribute data storage +- **Multiple OpenResty instances** - Load balance incoming requests +- **Stateless Node.js** - Run multiple API instances +- **Replace Unix socket** - Use TCP/HTTP for cross-host communication +- **Separate cert management** - Dedicated service for wildcard SSL + +## Monitoring and Observability + +### Logs + +- **OpenResty**: `/var/log/nginx/access.log`, `/var/log/nginx/error.log` +- **Node.js**: `journalctl -u proxy.service` +- **Redis**: `redis-cli MONITOR` + +### Health Checks + +- Node.js API: `curl http://localhost:3000/api/host` +- Redis: `redis-cli PING` +- OpenResty: `systemctl status openresty` +- Unix socket: `ls -la /var/run/proxy_lookup.socket` + +### Metrics to Monitor + +- Request rate and response times +- SSL certificate expiration dates +- Redis memory usage +- Host lookup cache hit rate +- Background service execution times + +[← Back to Home](index.html) diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..76ef5a3 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,308 @@ +--- +layout: default +title: Contributing +--- + +# Contributing Guide + +[← Back to Home](index.html) + +Thank you for considering contributing to the Proxy project! This guide will help you get started. + +## Development Setup + +### Prerequisites + +- Node.js 18+ (18.x, 20.x, or 22.x recommended) +- Redis server +- Git + +### Local Development + +1. **Clone the repository** + ```bash + git clone https://github.com/theta42/proxy.git + cd proxy/nodejs + ``` + +2. **Install dependencies** + ```bash + npm install + ``` + +3. **Start Redis** (if not already running) + ```bash + redis-server + ``` + +4. **Run in development mode** + ```bash + npm run dev + ``` + + This starts the Node.js API with nodemon for auto-reload on file changes. + +5. **Access the API** + - API: `http://localhost:3000/api` + - Web UI: `http://localhost:3000` + +## Testing + +The project uses Node.js built-in test runner (requires Node 18+). + +### Running Tests + +```bash +# Run all tests +npm test + +# Run only unit tests +npm run test:unit + +# Run only integration tests +npm run test:integration + +# Watch mode for development +npm run test:watch +``` + +### Test Structure + +``` +test/ +├── unit/ # Unit tests for isolated components +│ ├── callback_queue.test.js +│ ├── host_lookup.test.js +│ └── unix_socket.test.js +├── integration/ # Integration tests +│ └── dns_provider.test.js +└── helpers/ # Test utilities + └── dns_provider_contract.js +``` + +### Writing Tests + +We test **custom logic**, not third-party libraries: + +**DO test:** +- Host lookup algorithm +- Socket buffering logic +- DNS provider contracts +- Custom utility functions + +**DON'T test:** +- Express.js routing +- Redis ORM +- External DNS APIs (use mocks instead) + +### Adding DNS Provider Tests + +When adding a new DNS provider, you **must** add contract tests: + +```javascript +describe('NewProvider Provider', () => { + const NewProvider = require('../../models/dns_provider/newprovider'); + + test('should meet DNS provider contract', () => { + const mockCredentials = {api_key: 'mock-key'}; + const instance = validateDnsProviderContract(NewProvider, mockCredentials); + assert.ok(instance); + }); + + test('should have valid method signatures', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateMethodSignatures(instance); + }); + + test('should validate key mapping', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateKeyMapping(instance); + }); + + test('should validate type checking', () => { + const instance = new NewProvider({api_key: 'mock'}); + validateTypeChecking(instance); + }); +}); +``` + +See `test/integration/dns_provider.test.js` for examples. + +## Code Style + +### General Guidelines + +- Use strict mode: `'use strict';` +- Use tabs for indentation +- Clear, descriptive variable names +- Comment complex logic +- No trailing whitespace + +### File Organization + +```javascript +'use strict'; + +// 1. Node.js built-ins +const fs = require('fs'); +const path = require('path'); + +// 2. Third-party modules +const express = require('express'); +const redis = require('redis'); + +// 3. Local modules +const {Host} = require('./models'); +const middleware = require('./middleware/auth'); + +// 4. Code... +``` + +### Naming Conventions + +- Classes: `PascalCase` +- Functions: `camelCase` +- Constants: `UPPER_SNAKE_CASE` +- Private methods: `__privateMethod` (double underscore prefix) + +## Project Structure + +Understanding the codebase: + +``` +nodejs/ +├── models/ # Data models (Host, User, DNS providers) +├── routes/ # API route handlers +├── services/ # Background services (lookup, scheduler) +├── middleware/ # Express middleware +├── utils/ # Utility functions +├── public/ # Static web assets +├── views/ # EJS templates +└── test/ # Test suite +``` + +## Pull Request Process + +### Before Submitting + +1. **Run tests** - Ensure all tests pass + ```bash + npm test + ``` + +2. **Test locally** - Verify your changes work + ```bash + npm run dev + ``` + +3. **Update documentation** - Keep docs in sync with code changes + +4. **Commit messages** - Use clear, descriptive messages + ``` + Add DNS provider for Route53 + + - Implement Route53 DNS API client + - Add contract tests for Route53 + - Update documentation with Route53 setup + ``` + +### Submitting a PR + +1. **Fork the repository** + +2. **Create a feature branch** + ```bash + git checkout -b feature/my-new-feature + ``` + +3. **Make your changes** + +4. **Commit your changes** + ```bash + git add . + git commit -m "Description of changes" + ``` + +5. **Push to your fork** + ```bash + git push origin feature/my-new-feature + ``` + +6. **Open a Pull Request** on GitHub + +### PR Requirements + +- ✅ All tests must pass (CI/CD runs automatically) +- ✅ Tests run on Node.js 18.x, 20.x, and 22.x +- ✅ No merge conflicts with `master` +- ✅ Code follows project conventions +- ✅ New features include tests +- ✅ Documentation updated if needed + +### CI/CD Process + +When you open a PR: +1. GitHub Actions automatically runs tests +2. Tests execute on multiple Node.js versions +3. PR cannot be merged until all checks pass +4. Review from maintainers +5. Merge to master + +## Adding Features + +### Adding a DNS Provider + +1. **Create provider file** in `models/dns_provider/yourprovider.js` + +2. **Extend DnsApi base class** + ```javascript + const {DnsApi} = require('./common'); + + class YourProvider extends DnsApi { + static _keyMap = { + api_key: {isRequired: true, type: 'string', isPrivate: true} + }; + + // Implement required methods + async listDomains() { } + async getRecords(domain, options) { } + async createRecord(domain, options) { } + async deleteRecords(domain, options) { } + } + ``` + +3. **Add to provider list** in `models/dns_provider.js` + +4. **Add contract tests** in `test/integration/dns_provider.test.js` + +5. **Test your provider** + ```bash + npm run test:integration + ``` + +### Adding API Endpoints + +1. **Add route** in appropriate file (`routes/`) +2. **Update API documentation** (`nodejs/api.md`) +3. **Test the endpoint** manually and add integration tests if needed + +## Getting Help + +- **Questions?** Open a [GitHub Discussion](https://github.com/theta42/proxy/discussions) +- **Bug reports** Use [GitHub Issues](https://github.com/theta42/proxy/issues) +- **Security issues** Email maintainers directly (see package.json) + +## Code of Conduct + +- Be respectful and inclusive +- Focus on constructive feedback +- Help others learn and grow +- Follow the project's technical direction + +## License + +By contributing, you agree that your contributions will be licensed under the MIT License. + +--- + +[← Back to Home](index.html) | [View on GitHub](https://github.com/theta42/proxy) diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..f607f5b --- /dev/null +++ b/docs/index.md @@ -0,0 +1,97 @@ +--- +layout: default +title: Home +--- + +# Proxy + +A reverse proxy and HTTPS termination service using OpenResty/nginx with a management API and web GUI. + +## Features + +- **Automated HTTPS/SSL** - Let's Encrypt integration with HTTP-01 and DNS-01 challenges +- **Wildcard SSL Certificates** - Support for wildcard domains with automatic renewal +- **Multiple DNS Providers** - CloudFlare, DigitalOcean, PorkBun integrations +- **Advanced Routing** - Sophisticated wildcard domain matching (*, **) +- **RESTful API** - Full programmatic control +- **Web Interface** - User-friendly management GUI +- **High Performance** - Unix socket-based host lookup for minimal latency + +## Quick Start + +### Automated Installation + +For modern Debian-based systems (Ubuntu 20.04+, Debian 11+): + +```bash +wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash +``` + +### Requirements + +- Node.js 18+ (tested with 18.x, 20.x, 22.x) +- OpenResty (nginx with Lua support) +- Redis +- Linux system with root access + +## Documentation + +- [Installation Guide](installation.html) - Detailed setup instructions +- [API Reference](api.html) - Complete API documentation +- [Architecture](architecture.html) - System design and components +- [Contributing](contributing.html) - Development and testing guide + +## Use Cases + +**Development Teams** +- Host multiple projects on a single server with unique domains +- Automatic SSL for all development sites +- Easy configuration via API or web UI + +**Production Deployments** +- High-performance reverse proxy for microservices +- Centralized SSL certificate management +- Dynamic routing without nginx reloads + +**Personal Projects** +- Self-hosted services with automatic HTTPS +- Wildcard certificates for unlimited subdomains +- Simple management interface + +## Architecture + +``` +┌─────────────┐ +│ Client │ +└──────┬──────┘ + │ HTTPS + ▼ +┌─────────────────────┐ +│ OpenResty/Nginx │ +│ - SSL Termination │ +│ - Host Routing │ +└──────┬──────────────┘ + │ Unix Socket + ▼ +┌─────────────────────┐ ┌─────────────┐ +│ Node.js API │◄────►│ Redis │ +│ - Management │ │ - Storage │ +│ - SSL Orchestration│ │ - Cache │ +└──────┬──────────────┘ └─────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Backend Services │ +│ - Your Apps │ +└─────────────────────┘ +``` + +## Community + +- [GitHub Repository](https://github.com/theta42/proxy) +- [Issue Tracker](https://github.com/theta42/proxy/issues) +- [Pull Requests](https://github.com/theta42/proxy/pulls) + +## License + +MIT License - See [LICENSE](https://github.com/theta42/proxy/blob/master/LICENSE) for details. diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 0000000..b05f6b7 --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,221 @@ +--- +layout: default +title: Installation +--- + +# Installation Guide + +[← Back to Home](index.html) + +## Quick Install (Recommended) + +For modern Debian-based systems (Ubuntu 20.04+, Debian 11+): + +```bash +wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash +``` + +This automated installer will: +- Install Node.js 20.x +- Install OpenResty and required dependencies +- Install and configure Redis +- Set up SSL fallback certificates +- Install Lua dependencies +- Clone and install the proxy application +- Configure systemd service +- Start the proxy service + +## Manual Installation + +### System Requirements + +- Modern Linux distribution (Ubuntu 20.04+, Debian 11+, or equivalent) +- Root access +- Inbound internet access for Let's Encrypt validation +- Minimum 1GB RAM, 10GB disk space + +### Step 1: Install Dependencies + +**Ubuntu/Debian:** +```bash +apt install libpam0g-dev build-essential redis-server luarocks -y +``` + +### Step 2: Install Node.js 20.x + +```bash +curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \ + sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg + +NODE_MAJOR=20 +echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | \ + sudo tee /etc/apt/sources.list.d/nodesource.list + +apt update && apt install nodejs -y +``` + +Verify installation: +```bash +node --version # Should show v20.x.x +npm --version +``` + +### Step 3: Install OpenResty + +```bash +wget -O - https://openresty.org/package/pubkey.gpg | \ + sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg + +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | \ + sudo tee /etc/apt/sources.list.d/openresty.list + +apt update && apt install openresty -y +``` + +### Step 4: Install Lua Dependencies + +```bash +luarocks install lua-resty-auto-ssl +luarocks install luasocket +``` + +### Step 5: SSL Configuration + +Create fallback SSL certificates: + +```bash +mkdir -p /etc/ssl/ + +openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \ + -subj '/CN=sni-support-required-for-valid-ssl' \ + -keyout /etc/ssl/resty-auto-ssl-fallback.key \ + -out /etc/ssl/resty-auto-ssl-fallback.crt +``` + +### Step 6: Configure OpenResty + +Clone the repository and copy configuration files: + +```bash +cd /var/www +git clone https://github.com/theta42/proxy.git +cd proxy + +# Copy nginx configs +mkdir -p /etc/openresty/sites-enabled/ +cp ops/nginx_conf/nginx.conf /etc/openresty/nginx.conf +cp ops/nginx_conf/autossl.conf /etc/openresty/autossl.conf +cp ops/nginx_conf/proxy.conf /etc/openresty/sites-enabled/000-proxy +cp ops/nginx_conf/targetinfo.lua /usr/local/openresty/lualib/targetinfo.lua +``` + +### Step 7: Install Application + +```bash +cd /var/www/proxy/nodejs +npm install +``` + +### Step 8: Configure Systemd Service + +```bash +cp /var/www/proxy/ops/proxy.service /etc/systemd/system/proxy.service +systemctl daemon-reload +systemctl enable proxy.service +systemctl start proxy.service +``` + +Verify service is running: +```bash +systemctl status proxy.service +``` + +### Step 9: Initial Setup + +The proxy API will be available on port 3000 by default. You'll need to: + +1. Create your first user account +2. Configure DNS providers (for wildcard SSL) +3. Add your first host + +See the [API Reference](api.html) for details. + +## Configuration + +### Environment Variables + +- `NODE_ENV` - Set to `production` for production deployments +- `NODE_PORT` - Override default port (default: 3000) + +### Redis Configuration + +The proxy uses Redis with the prefix `proxy_`. To change this, edit `nodejs/conf/base.js`: + +```javascript +redis: { + prefix: 'proxy_' +} +``` + +### OpenResty Configuration + +Key configuration files in `/etc/openresty/`: +- `nginx.conf` - Main nginx configuration +- `autossl.conf` - Let's Encrypt HTTP-01 challenge handler +- `sites-enabled/000-proxy` - Proxy server configuration + +### Unix Socket + +The proxy communicates with OpenResty via Unix socket at: +``` +/var/run/proxy_lookup.socket +``` + +This path is configurable in `nodejs/conf/base.js`. + +## Troubleshooting + +### Service won't start + +Check logs: +```bash +journalctl -u proxy.service -f +``` + +Common issues: +- Port 3000 already in use +- Redis not running: `systemctl status redis-server` +- Permission issues: Service must run as root for user management + +### SSL certificates not working + +Check OpenResty logs: +```bash +tail -f /var/log/nginx/error.log +``` + +Common issues: +- Firewall blocking ports 80/443 +- DNS not pointing to server +- Let's Encrypt rate limits exceeded + +### Host lookup not working + +Check Unix socket: +```bash +ls -la /var/run/proxy_lookup.socket +# Should show srwxrwxrwx (socket permissions) +``` + +Test lookup: +```bash +echo '{"domain":"example.com"}' | nc -U /var/run/proxy_lookup.socket +``` + +## Next Steps + +- [Configure DNS Providers](api.html#dns-providers) for wildcard SSL +- [Add your first host](api.html#hosts) +- [Set up the web interface](index.html) + +[← Back to Home](index.html)