Merge pull request #83 from theta42/DNS-01

Dns 01
This commit is contained in:
2025-12-31 17:33:16 -05:00
committed by GitHub
41 changed files with 4964 additions and 1389 deletions
+83
View File
@@ -0,0 +1,83 @@
# CI/CD Workflows
## Pull Request Testing
The `pr-tests.yml` workflow automatically runs on every pull request to the `master` branch.
### What it does:
1. **Multi-version testing**: Tests run on Node.js 18.x, 20.x, and 22.x
2. **Comprehensive coverage**: Runs unit tests, integration tests, and full test suite
3. **Blocks merging**: PRs cannot be merged until all tests pass on all Node versions
### Workflow triggers:
- Opening a pull request to `master`
- Pushing new commits to an existing PR
- Updates to PR branches
### Test jobs:
1. **Unit tests** - Tests isolated components (callback queue, host lookup, unix socket)
2. **Integration tests** - Tests DNS provider contracts
3. **Full test suite** - Complete test coverage
### Branch protection:
The `master` branch is protected and requires:
- All tests must pass before merging
- Status check: `test` job must succeed
- Applies to all contributors (admins can override)
## Running tests locally:
Before creating a PR, run tests locally to catch issues early:
```bash
cd nodejs
npm run test:unit # Run unit tests only
npm run test:integration # Run integration tests only
npm test # Run all tests
npm run test:watch # Watch mode for development
```
## Adding new workflows:
To add new CI/CD workflows:
1. Create a new `.yml` file in `.github/workflows/`
2. Define triggers, jobs, and steps
3. Test the workflow by creating a PR
4. Add status check to branch protection if required for merging
## Troubleshooting:
**Tests pass locally but fail in CI:**
- Check Node.js version compatibility (workflow tests 18.x, 20.x, 22.x)
- Verify all dependencies are in package.json (not installed globally)
- Check for environment-specific issues (paths, permissions)
**Branch protection preventing merge:**
- Ensure all required status checks pass
- Check workflow logs for detailed error messages
- Re-run failed jobs if transient failures occurred
**Modifying branch protection:**
```bash
# View current protection settings
gh api repos/theta42/proxy/branches/master/protection
# Update required status checks
gh api repos/theta42/proxy/branches/master/protection \
-X PUT \
-F required_status_checks[contexts][]=test \
-F required_status_checks[contexts][]=your-new-check
```
## Workflow status:
View workflow runs and status:
- GitHub UI: https://github.com/theta42/proxy/actions
- CLI: `gh run list`
- PR checks: Automatically shown on PR page
+61
View File
@@ -0,0 +1,61 @@
name: Pull Request Tests
# Run tests on pull requests to master and when pushing to PRs
on:
pull_request:
branches:
- master
push:
branches-ignore:
- master
jobs:
test:
name: Run Tests
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18.x, 20.x, 22.x]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: nodejs/package-lock.json
- name: Install dependencies
working-directory: ./nodejs
run: npm ci
- name: Run unit tests
working-directory: ./nodejs
run: npm run test:unit
- name: Run integration tests
working-directory: ./nodejs
run: npm run test:integration
- name: Run all tests
working-directory: ./nodejs
run: npm test
test-summary:
name: Test Summary
runs-on: ubuntu-latest
needs: test
if: always()
steps:
- name: Check test results
run: |
if [ "${{ needs.test.result }}" != "success" ]; then
echo "Tests failed. PR cannot be merged."
exit 1
fi
echo "All tests passed successfully!"
+197 -136
View File
@@ -1,157 +1,218 @@
# proxy
# Proxy
A simple reverse proxy and https termination using openresty/nginx with a managment API and GUI.
A reverse proxy and HTTPS termination service using OpenResty/nginx with a management API and web GUI.
## API docs
[API docs](api.md)
**Documentation:** [https://theta42.github.io/proxy/](https://theta42.github.io/proxy/)
## Server set up
## Features
The server requires:
* NodeJS 8.x
* inbound Internet access
* OpenResty
* redis
* lua rocks
- Automated HTTPS/SSL certificate management via Let's Encrypt
- Support for HTTP-01 (auto-ssl) and DNS-01 (wildcard) ACME challenges
- Multiple DNS provider integrations (CloudFlare, DigitalOcean, PorkBun)
- Wildcard SSL certificate support with automatic renewal
- Dynamic host routing with wildcard domain matching (*, **)
- Web-based management interface
- RESTful API for automation
- User authentication and management
- Unix socket-based host lookup for high-performance routing
This has been tested on ubuntu 16.04, but should work on any modern Linux
distro.
**Optional** Linux users for its user management, so this will
**ONLY** work on Linux, no macOS, BSD or Windows and require root.
## Requirements
The steps below are for a new ubuntu server, they should be mostly the same for
other distros, but the paths and availability of packages may vary. A dedicated
server is highly recommended (since it will make ever user a system user), a VPS
like Digital Ocean will do just fine.
- Node.js 18+ (tested with 18.x, 20.x, 22.x)
- OpenResty (nginx with Lua support)
- Redis
- Modern Linux distribution (tested on Ubuntu 20.04+, Debian 11+)
- Inbound internet access for Let's Encrypt validation
- Root access (required for user management features)
* Install openresty
## Quick Install
[OpenResty® Linux Packages](https://openresty.org/en/linux-packages.html)
* These packages are needed for the PAM node package
```bash
apt install libpam0g-dev build-essential
```
* Install redis
```bash
apt install redis-server
```
* install lua plugin
An automated installer is available for modern Debian-based systems:
```bash
apt install luarocks
sudo luarocks install lua-resty-auto-ssl
sudo luarocks install lua-resty-socket
sudo luarocks install lua-socket
sudo luarocks install socket
sudo luarocks install luasocket
sudo luarocks install luasocket-unix
sudo luarocks install lua-cjson
wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash
```
* openresty config
This installer will:
- Install Node.js 20.x
- Install OpenResty and required dependencies
- Install and configure Redis
- Set up SSL fallback certificates
- Install Lua dependencies (lua-resty-auto-ssl, luasocket)
- Clone and install the proxy application
- Configure systemd service
- Start the proxy service
Set up fail back SSL certs
## Manual Installation
For manual installation or other distributions, see the detailed steps below.
### System Dependencies
**Ubuntu/Debian:**
```bash
mkdir /etc/ssl/
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj '/CN=sni-support-required-for-valid-ssl' -keyout /etc/ssl/resty-auto-ssl-fallback.key -out /etc/ssl/resty-auto-ssl-fallback.crt
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -subj '/CN=sni-support-required-for-valid-ssl' -keyout /etc/ssl/resty-auto-ssl-fallback.key -out /etc/ssl/resty-auto-ssl-fallback.crt
# openssl dhparam -out /etc/nginx/dhparam.pem 4096 # This takes a LONG time and is not needed.
apt install libpam0g-dev build-essential redis-server luarocks -y
```
Change the `/etc/openresty/nginx.conf to have this config`
```
#user nobody;
worker_processes 4;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
client_max_body_size 4g;
lua_shared_dict auto_ssl 100m;
lua_shared_dict auto_ssl_settings 64k;
resolver 8.8.4.4 8.8.8.8;
init_by_lua_block {
auto_ssl = (require "resty.auto-ssl").new()
auto_ssl:set("storage_adapter", "resty.auto-ssl.storage_adapters.redis")
auto_ssl:set("allow_domain", function(domain)
return true
end)
auto_ssl:init()
}
init_worker_by_lua_block {
auto_ssl:init_worker()
}
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
server {
listen 127.0.0.1:8999;
# Increase the body buffer size, to ensure the internal POSTs can always
# parse the full POST contents into memory.
client_body_buffer_size 128k;
client_max_body_size 128k;
location / {
content_by_lua_block {
auto_ssl:hook_server()
}
}
}
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
include sites-enabled/*;
}
**Node.js 20.x:**
```bash
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
NODE_MAJOR=20
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
apt update && apt install nodejs -y
```
add the SSL config file `/etc/openresty/autossl.conf`, contents from here
https://github.com/theta42/t42-common/blob/master/templates/openresty/autossl.conf.erb
**OpenResty:**
```bash
wget -O - https://openresty.org/package/pubkey.gpg | sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/openresty.list
apt update && apt install openresty -y
```
**Lua Dependencies:**
```bash
luarocks install lua-resty-auto-ssl
luarocks install luasocket
```
Add the proxy config `/etc/openresty/sites-enabled/000-proxy` contents from here
https://github.com/theta42/t42-common/blob/master/templates/openresty/010-proxy.conf.erb
### SSL Configuration
Create fallback SSL certificates:
```bash
mkdir -p /etc/ssl/
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \
-subj '/CN=sni-support-required-for-valid-ssl' \
-keyout /etc/ssl/resty-auto-ssl-fallback.key \
-out /etc/ssl/resty-auto-ssl-fallback.crt
```
### OpenResty Configuration
Configuration files are provided in `ops/nginx_conf/`:
- `nginx.conf` - Main nginx configuration
- `autossl.conf` - Auto-SSL configuration for Let's Encrypt HTTP-01
- `proxy.conf` - Proxy server configuration with host lookup
- `targetinfo.lua` - Lua module for host lookup via Unix socket
Copy these files to `/etc/openresty/`:
```bash
mkdir -p /etc/openresty/sites-enabled/
cp ops/nginx_conf/nginx.conf /etc/openresty/nginx.conf
cp ops/nginx_conf/autossl.conf /etc/openresty/autossl.conf
cp ops/nginx_conf/proxy.conf /etc/openresty/sites-enabled/000-proxy
cp ops/nginx_conf/targetinfo.lua /usr/local/openresty/lualib/targetinfo.lua
```
### Application Setup
Clone and install:
```bash
cd /var/www
git clone https://github.com/theta42/proxy.git
cd proxy/nodejs
npm install
```
Create systemd service:
```bash
cp ops/proxy.service /etc/systemd/system/proxy.service
systemctl daemon-reload
systemctl enable proxy.service
systemctl start proxy.service
```
## DNS Provider Configuration
For wildcard SSL certificates, configure a DNS provider via the web UI or API:
**Supported providers:**
- **CloudFlare** - Requires API token
- **DigitalOcean** - Requires API token
- **PorkBun** - Requires API key and secret API key
Once configured, create a wildcard host (e.g., `*.example.com`) and the system will automatically request and manage the DNS-01 challenge certificate.
## Architecture
The system consists of three main components:
1. **OpenResty/Nginx** - Frontend proxy with Lua-based routing
- Handles SSL termination via lua-resty-auto-ssl
- Queries Node.js backend via Unix socket for host routing
- Proxies requests to configured backend servers
2. **Node.js API** - Backend management and control plane
- RESTful API for host/user/DNS management
- Wildcard SSL certificate orchestration
- Host lookup tree with wildcard matching
- User authentication and authorization
3. **Redis** - Data store
- Host configurations
- User accounts and tokens
- SSL certificate storage
- Domain and DNS provider configurations
## Host Lookup System
The proxy supports sophisticated domain matching:
- **Exact match**: `example.com` matches only `example.com`
- **Single wildcard**: `*.example.com` matches `sub.example.com` but not `deep.sub.example.com`
- **Double wildcard**: `**.example.com` matches any depth (`sub.example.com`, `deep.sub.example.com`, etc.)
- **Mixed wildcards**: `api.*.example.com` matches `api.v1.example.com`, `api.v2.example.com`, etc.
Priority: Exact match > Single wildcard > Double wildcard
## Development
**Running locally:**
```bash
cd nodejs
npm install
npm run dev # Runs with nodemon for auto-reload
```
**Running tests:**
```bash
npm test # Run all tests
npm run test:unit # Run unit tests only
npm run test:watch # Watch mode for development
```
Tests use Node.js built-in test runner (requires Node 18+).
## API Documentation
See [API Documentation](nodejs/api.md) for complete API reference.
## Contributing
Pull requests are welcome. The project uses GitHub Actions for CI/CD:
- Tests run automatically on all PRs
- All tests must pass before merging to master
- Tests run on Node.js 18.x, 20.x, and 22.x
## License
MIT - See LICENSE file for details.
## Project Structure
```
proxy/
├── nodejs/ # Node.js backend application
│ ├── bin/ # Entry point (www)
│ ├── models/ # Data models (Host, User, DNS providers)
│ ├── routes/ # API routes
│ ├── services/ # Background services (host lookup, scheduler)
│ ├── middleware/ # Express middleware
│ ├── utils/ # Utility functions
│ ├── public/ # Static web assets
│ ├── views/ # EJS templates
│ └── test/ # Test suite
├── ops/ # Operations and deployment
│ ├── nginx_conf/ # OpenResty configuration files
│ ├── install.sh # Automated installer
│ └── proxy.service # Systemd service definition
└── .github/workflows/ # CI/CD workflows
```
+44
View File
@@ -0,0 +1,44 @@
# Documentation
This directory contains the GitHub Pages documentation site for the Proxy project.
**Live site:** https://theta42.github.io/proxy/
## Pages
- `index.md` - Home page with project overview
- `installation.md` - Installation and setup guide
- `api.md` - Complete API reference
- `architecture.md` - System architecture and design
- `contributing.md` - Development and contribution guide
## Local Preview
To preview the site locally:
```bash
# Install Jekyll (one-time setup)
gem install jekyll bundler
# Run local server
cd docs
jekyll serve
# View at http://localhost:4000/proxy/
```
## Theme
The site uses the Cayman theme (`jekyll-theme-cayman`). Configuration is in `_config.yml`.
## Updating Documentation
1. Edit markdown files in this directory
2. Commit and push to master branch
3. GitHub Pages automatically rebuilds (may take 1-2 minutes)
4. Changes visible at https://theta42.github.io/proxy/
## Legacy Documentation
- `dev_setup.md` - Old development setup notes (kept for reference)
- `Update 4.11.md` - Old update notes (kept for reference)
+9
View File
@@ -0,0 +1,9 @@
title: Proxy
description: A reverse proxy and HTTPS termination service using OpenResty/nginx with a management API and web GUI
theme: jekyll-theme-cayman
show_downloads: true
github:
repository_url: https://github.com/theta42/proxy
zip_url: https://github.com/theta42/proxy/archive/refs/heads/master.zip
tar_url: https://github.com/theta42/proxy/archive/refs/heads/master.tar.gz
repository_name: theta42/proxy
Executable
+549
View File
@@ -0,0 +1,549 @@
---
layout: default
title: API Reference
---
# API Documentation
[← Back to Home](index.html)
All API endpoints require authentication via the `auth-token` header unless otherwise noted.
Base URL: `https://your-proxy-host.com/api`
---
## Authentication
### Login
**POST** `/api/auth/login`
Authenticate a user and receive an auth token.
```bash
curl -H "Content-Type: application/json" \
-X POST \
-d '{"username": "myuser", "password": "mypassword"}' \
https://proxy-host.com/api/auth/login
```
**Responses:**
- `200` `{"login": true, "token": "027d3964-7d81-4462-a6f9-2c1f9b40b4be", "message": "myuser logged in!"}`
- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}`
### Logout
**ALL** `/api/auth/logout`
Invalidate the current auth token.
```bash
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/auth/logout
```
**Responses:**
- `200` `{"message": "Bye"}`
---
## Users
All user endpoints require authentication.
### List Users
**GET** `/api/user`
Get list of all users.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/user
```
**Query Parameters:**
- `detail` - Include full user details (optional)
**Responses:**
- `200` `{"results": ["user1", "user2"]}`
- `200` `{"results": [{"username": "user1", ...}, ...]}` (with `?detail=true`)
### Get Current User
**GET** `/api/user/me`
Get information about the currently authenticated user.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/user/me
```
**Responses:**
- `200` `{"username": "myuser"}`
### Create User
**POST** `/api/user`
Create a new user.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"username": "newuser", "password": "newpassword"}' \
https://proxy-host.com/api/user
```
**Responses:**
- `200` User created successfully
- `409` Username already exists
- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error
### Delete User
**DELETE** `/api/user/:username`
Delete a user account.
```bash
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/user/olduser
```
**Responses:**
- `200` `{"username": "olduser", "results": ...}`
- `404` User not found
### Change Password (Self)
**PUT** `/api/user/password`
Change the password for the currently authenticated user.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"password": "newpassword"}' \
https://proxy-host.com/api/user/password
```
**Responses:**
- `200` `{"results": ...}` Password changed successfully
### Change Password (Other User)
**PUT** `/api/user/password/:username`
Change the password for another user (admin function).
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"password": "newpassword"}' \
https://proxy-host.com/api/user/password/otheruser
```
**Responses:**
- `200` `{"results": ...}` Password changed successfully
- `404` User not found
### Create Invite Token
**POST** `/api/user/invite`
Create an invitation token for new user registration.
```bash
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/user/invite
```
**Responses:**
- `200` `{"token": "5caf94d2-2c91-4010-8df7-968d10802b9d"}`
### Add SSH Key
**POST** `/api/user/key`
Add an SSH public key to the current user's account.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"key": "ssh-rsa AAAAB3..."}' \
https://proxy-host.com/api/user/key
```
**Responses:**
- `200` `{"message": true}` Key added successfully
- `400` `{"message": "Bad SSH key"}` Invalid key format
---
## Hosts
Manage proxy host configurations.
### List Hosts
**GET** `/api/host`
Get list of all configured hosts.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host
```
**Query Parameters:**
- `detail` - Include full host details (optional)
**Responses:**
- `200` `{"results": ["example.com", "*.wildcard.com"]}`
- `200` `{"results": [{"host": "example.com", "ip": "192.168.1.10", ...}, ...]}` (with `?detail=true`)
### Get Host
**GET** `/api/host/:host`
Get configuration for a specific host.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"item": "example.com", "results": {"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, ...}}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
### Lookup Host
**GET** `/api/host/lookup/:domain`
Test the host lookup algorithm (supports wildcard matching).
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/lookup/sub.example.com
```
**Responses:**
- `200` `{"string": "sub.example.com", "results": {"host": "*.example.com", ...}}`
- `200` `{"string": "sub.example.com", "results": null}` (no match)
### Get Lookup Tree
**GET** `/api/host/lookupobj`
Get the internal lookup tree structure (for debugging).
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/lookupobj
```
**Responses:**
- `200` `{"results": {"com": {"example": {...}}}}`
### Create Host
**POST** `/api/host`
Add a new host configuration.
**Parameters:**
- `host` (required) - Domain name (e.g., `example.com`, `*.example.com`)
- `ip` (required) - Target IP address or FQDN
- `targetPort` (required) - Target port number (1-65535)
- `forcessl` (optional) - Force HTTPS redirect (default: true)
- `targetssl` (optional) - Use HTTPS to backend (default: false)
- `challengeType` (optional) - For wildcards: `DNS-01-wildcard` or `wildcardChild`
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, "forcessl": true, "targetssl": false}' \
https://proxy-host.com/api/host
```
**Responses:**
- `200` `{"message": "\"example.com\" added.", "host": "example.com", ...}`
- `409` `{"name": "HostNameUsed", "message": "Host already exists"}`
- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error
### Update Host
**PUT** `/api/host/:host`
Update an existing host configuration.
**Parameters:** Same as Create Host (all optional)
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"ip": "192.168.1.20", "targetPort": 9000}' \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"message": "\"example.com\" updated.", ...}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
- `422` Validation error
### Delete Host
**DELETE** `/api/host/:host`
Remove a host configuration.
```bash
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"message": "example.com deleted", ...}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
### Renew Wildcard Certificate
**PUT** `/api/host/:host/renew`
Manually trigger wildcard certificate renewal.
```bash
curl -H "auth-token: your-token-here" \
-X PUT \
https://proxy-host.com/api/host/*.example.com/renew
```
**Responses:**
- `200` `{"message": "Requesting wildcard cert for *.example.com"}`
- `404` Host not found
---
## DNS Providers
Manage DNS provider integrations for wildcard SSL certificates.
### List DNS Providers
**GET** `/api/dns`
Get list of configured DNS providers.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns
```
**Query Parameters:**
- `detail` - Include full provider details (optional)
**Responses:**
- `200` `{"results": ["provider-id-1", "provider-id-2"]}`
### List Available Provider Types
**OPTIONS** `/api/dns`
Get list of supported DNS provider types and their configuration requirements.
```bash
curl -H "auth-token: your-token-here" \
-X OPTIONS \
https://proxy-host.com/api/dns
```
**Responses:**
- `200` `{"results": [{"name": "CloudFlare", "fields": {...}}, {"name": "DigitalOcean", ...}, ...]}`
### Create DNS Provider
**POST** `/api/dns`
Configure a new DNS provider.
**CloudFlare:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My CloudFlare", "dnsProvider": "Cloudflare", "token": "your-api-token"}' \
https://proxy-host.com/api/dns
```
**DigitalOcean:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My DO", "dnsProvider": "DigitalOcean", "token": "your-api-token"}' \
https://proxy-host.com/api/dns
```
**PorkBun:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My PorkBun", "dnsProvider": "PorkBun", "apiKey": "pk_xxx", "secretApiKey": "sk_xxx"}' \
https://proxy-host.com/api/dns
```
**Responses:**
- `200` `{"message": "\"provider-id\" added.", ...}`
- `422` Validation error or invalid API credentials
### Get DNS Provider
**GET** `/api/dns/:id`
Get a specific DNS provider configuration.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"item": "provider-id", "results": {...}}`
- `404` Provider not found
### Update DNS Provider
**PUT** `/api/dns/:id`
Update DNS provider configuration.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"name": "Updated Name"}' \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"message": "\"provider-id\" updated.", ...}`
- `404` Provider not found
### Delete DNS Provider
**DELETE** `/api/dns/:id`
Remove a DNS provider and all associated domains.
```bash
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"message": "provider-id deleted", ...}`
- `404` Provider not found
### List Domains
**GET** `/api/dns/domain`
List all domains from all configured providers.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/domain
```
**Query Parameters:**
- `detail` - Include full domain details (optional)
**Responses:**
- `200` `{"results": ["example.com", "test.com"]}`
### Get Domain
**GET** `/api/dns/domain/:domain`
Get details for a specific domain.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/domain/example.com
```
**Responses:**
- `200` `{"results": [{"domain": "example.com", "zoneId": "...", ...}]}`
- `404` Domain not found
### Refresh Domains
**POST** `/api/dns/domain/refresh/:providerId`
Refresh the domain list from a DNS provider's API.
```bash
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/dns/domain/refresh/provider-id
```
**Responses:**
- `200` `{"results": ...}` Updated domain list
- `404` Provider not found
---
## Certificates
Retrieve SSL certificate information.
### Get Certificate
**GET** `/api/cert/:host`
Get the SSL certificate for a host.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/cert/example.com
```
**Responses:**
- `200` Certificate data including `cert_pem`, `fullchain_pem`, `privkey_pem`, expiry information
- `404` Certificate not found
---
## Error Responses
All endpoints may return the following error responses:
- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - Authentication required or invalid
- `404` `{"name": "NotFound", "message": "..."}` - Resource not found
- `422` `{"name": "ObjectValidateError", "message": [...], "keys": [...]}` - Validation errors
- `500` Internal server error
## Notes
- All timestamps are in milliseconds since epoch
- The `auth-token` header is required for all authenticated endpoints
- Host names support wildcards: `*` (single level) and `**` (multi-level)
- DNS providers are validated on creation - invalid API credentials will be rejected
- Wildcard certificates are automatically renewed 30 days before expiration
+265
View File
@@ -0,0 +1,265 @@
---
layout: default
title: Architecture
---
# Architecture
[← Back to Home](index.html)
## System Overview
The proxy system consists of three main components working together to provide high-performance reverse proxying with automated SSL management.
```
┌──────────────────────────────────────────────────────────────┐
│ Internet │
└─────────────────────────┬────────────────────────────────────┘
│ HTTPS/HTTP
┌──────────────────────────────────────────────────────────────┐
│ OpenResty/Nginx │
│ ┌────────────────┐ ┌──────────────┐ ┌─────────────────┐ │
│ │ SSL Termination│ │ Host Routing │ │ Request Proxying│ │
│ │ (lua-resty- │ │ (targetinfo. │ │ │ │
│ │ auto-ssl) │ │ lua) │ │ │ │
│ └────────────────┘ └──────────────┘ └─────────────────┘ │
└────────────┬──────────────────┬───────────────────────────┬──┘
│ │ │
Let's Encrypt Unix Socket Backend
HTTP-01 Lookup Query Services
│ │ │
▼ ▼ ▼
┌──────────────────────────────────────────────────────────────┐
│ Node.js Application │
│ ┌──────────────┐ ┌────────────────┐ ┌─────────────────┐ │
│ │ Services │ │ Models │ │ Routes │ │
│ │ - host_lookup│ │ - Host │ │ - /api/host │ │
│ │ - scheduler │ │ - DNS Provider │ │ - /api/dns │ │
│ └──────────────┘ │ - User │ │ - /api/user │ │
│ │ - Auth │ │ - /api/auth │ │
│ └────────────────┘ │ - /api/cert │ │
│ └─────────────────┘ │
└────────────┬────────────────────┬────────────────────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ Redis │ │ DNS Providers │
│ - Host configs │ │ - CloudFlare │
│ - User accounts │ │ - DigitalOcean │
│ - SSL certs │ │ - PorkBun │
│ - Auth tokens │ │ (DNS-01 challenges) │
└──────────────────────┘ └──────────────────────┘
```
## Component Details
### OpenResty/Nginx (Frontend)
**Responsibilities:**
- Accept incoming HTTP/HTTPS requests
- SSL termination using lua-resty-auto-ssl
- Host-based routing decisions
- Proxy requests to backend services
**Key Features:**
- HTTP-01 ACME challenge handling for automatic SSL
- Lua-based host lookup via Unix socket
- High-performance event-driven architecture
- Support for WebSocket connections
**Configuration Files:**
- `/etc/openresty/nginx.conf` - Main configuration
- `/etc/openresty/autossl.conf` - Let's Encrypt integration
- `/etc/openresty/sites-enabled/000-proxy` - Proxy configuration
- `/usr/local/openresty/lualib/targetinfo.lua` - Host lookup module
### Node.js Application (Backend)
**Responsibilities:**
- API for host/user/DNS management
- Wildcard SSL certificate orchestration
- Host lookup tree maintenance
- User authentication and authorization
**Directory Structure:**
```
nodejs/
├── bin/www # Application entry point
├── models/ # Data models
│ ├── host.js # Host configuration and lookup
│ ├── auth.js # Authentication logic
│ ├── user.js # User management
│ └── dns_provider/ # DNS provider implementations
├── routes/ # API endpoints
│ ├── host.js # Host CRUD operations
│ ├── dns.js # DNS provider management
│ ├── user.js # User management
│ └── auth.js # Authentication
├── services/ # Background services
│ ├── host_lookup.js # Unix socket server
│ └── host_scheduler.js # Cert renewal scheduler
├── middleware/ # Express middleware
│ └── auth.js # Authentication middleware
└── utils/ # Utility modules
└── unix_socket_json.js # Unix socket server
```
### Redis (Data Store)
**Stored Data:**
- Host configurations (domain, IP, port, SSL settings)
- User accounts and hashed passwords
- Authentication tokens
- SSL certificates (for wildcard domains)
- DNS provider credentials
- Domain-to-provider mappings
**Key Prefixes:**
```
proxy_Host_<hostname> # Host configuration
proxy_User_<username> # User account
proxy_AuthToken_<token> # Auth tokens
proxy_DnsProvider_<id> # DNS provider
proxy_Domain_<domain> # Domain info
<hostname>:latest # SSL certificate cache
```
## Request Flow
### Standard HTTP/HTTPS Request
1. **Client** sends HTTPS request to `app.example.com`
2. **OpenResty** receives request, terminates SSL
3. **Lua script** (`targetinfo.lua`) queries Redis for host config
4. If **cache miss**, Lua queries Node.js via Unix socket
5. **Node.js** performs host lookup (supports wildcards)
6. **Response** returned with target IP and port
7. **OpenResty** proxies request to backend service
8. **Response** proxied back to client
### Wildcard SSL Certificate Request
1. **User** creates wildcard host (`*.example.com`) via API
2. **Node.js** validates domain has DNS provider configured
3. **Let's Encrypt** DNS-01 challenge initiated
4. **DNS provider** API creates TXT record (`_acme-challenge.example.com`)
5. **Let's Encrypt** validates TXT record
6. **Certificate** generated and stored in Redis
7. **DNS provider** cleans up TXT record
8. **Background scheduler** monitors expiration, renews 30 days before expiry
## Host Lookup Algorithm
The lookup tree enables sophisticated domain matching:
```
Input: "api.v1.example.com"
Tree Structure:
{
"com": {
"example": {
"*": { // Matches api.example.com
"#record": {...}
},
"v1": {
"api": { // Matches api.v1.example.com (exact)
"#record": {...}
}
}
}
}
}
Priority: Exact > Single wildcard (*) > Double wildcard (**)
```
**Wildcard Types:**
- `example.com` - Exact match only
- `*.example.com` - Matches `sub.example.com` (single level)
- `**.example.com` - Matches any depth (`sub.deep.example.com`)
- `api.*.example.com` - Matches `api.v1.example.com`, `api.v2.example.com`
## Security Architecture
### Authentication Flow
1. User sends credentials to `/api/auth/login`
2. Credentials validated against stored hash (bcrypt)
3. Token generated and stored in Redis with TTL
4. Token returned to client
5. Subsequent requests include token in `auth-token` header
6. Middleware validates token before processing request
### SSL Certificate Security
- **Private keys** stored only in Redis (memory/disk based on config)
- **Fallback certificates** used when SNI unavailable
- **Let's Encrypt** rate limiting respected
- **DNS provider credentials** marked as `isPrivate` (not returned in API)
### Unix Socket Communication
- Socket file: `/var/run/proxy_lookup.socket`
- Permissions: `777` (container-safe, single-use deployment)
- Protocol: JSON over Unix stream socket
- Buffer handling: Accumulates partial messages until complete JSON
## Performance Optimizations
### Caching Strategy
1. **Redis cache** - Primary host configuration storage
2. **Lookup tree** - In-memory host lookup (rebuilt on changes)
3. **OpenResty cache** - Reduces Unix socket calls
4. **Wildcard parent caching** - Stores resolved wildcard parents
### Unix Socket vs HTTP API
Unix socket chosen over HTTP for host lookups:
- **Lower latency** - No TCP overhead
- **Higher throughput** - No HTTP parsing
- **Simpler** - Direct JSON communication
- **Secure** - Filesystem permissions, no network exposure
## Scalability Considerations
### Current Architecture
- **Single instance** - OpenResty + Node.js + Redis on one server
- **Vertical scaling** - Add CPU/RAM as needed
- **Limitations** - Unix socket ties OpenResty to Node.js on same host
### Future Scaling Options
- **Redis cluster** - Distribute data storage
- **Multiple OpenResty instances** - Load balance incoming requests
- **Stateless Node.js** - Run multiple API instances
- **Replace Unix socket** - Use TCP/HTTP for cross-host communication
- **Separate cert management** - Dedicated service for wildcard SSL
## Monitoring and Observability
### Logs
- **OpenResty**: `/var/log/nginx/access.log`, `/var/log/nginx/error.log`
- **Node.js**: `journalctl -u proxy.service`
- **Redis**: `redis-cli MONITOR`
### Health Checks
- Node.js API: `curl http://localhost:3000/api/host`
- Redis: `redis-cli PING`
- OpenResty: `systemctl status openresty`
- Unix socket: `ls -la /var/run/proxy_lookup.socket`
### Metrics to Monitor
- Request rate and response times
- SSL certificate expiration dates
- Redis memory usage
- Host lookup cache hit rate
- Background service execution times
[← Back to Home](index.html)
+308
View File
@@ -0,0 +1,308 @@
---
layout: default
title: Contributing
---
# Contributing Guide
[← Back to Home](index.html)
Thank you for considering contributing to the Proxy project! This guide will help you get started.
## Development Setup
### Prerequisites
- Node.js 18+ (18.x, 20.x, or 22.x recommended)
- Redis server
- Git
### Local Development
1. **Clone the repository**
```bash
git clone https://github.com/theta42/proxy.git
cd proxy/nodejs
```
2. **Install dependencies**
```bash
npm install
```
3. **Start Redis** (if not already running)
```bash
redis-server
```
4. **Run in development mode**
```bash
npm run dev
```
This starts the Node.js API with nodemon for auto-reload on file changes.
5. **Access the API**
- API: `http://localhost:3000/api`
- Web UI: `http://localhost:3000`
## Testing
The project uses Node.js built-in test runner (requires Node 18+).
### Running Tests
```bash
# Run all tests
npm test
# Run only unit tests
npm run test:unit
# Run only integration tests
npm run test:integration
# Watch mode for development
npm run test:watch
```
### Test Structure
```
test/
├── unit/ # Unit tests for isolated components
│ ├── callback_queue.test.js
│ ├── host_lookup.test.js
│ └── unix_socket.test.js
├── integration/ # Integration tests
│ └── dns_provider.test.js
└── helpers/ # Test utilities
└── dns_provider_contract.js
```
### Writing Tests
We test **custom logic**, not third-party libraries:
**DO test:**
- Host lookup algorithm
- Socket buffering logic
- DNS provider contracts
- Custom utility functions
**DON'T test:**
- Express.js routing
- Redis ORM
- External DNS APIs (use mocks instead)
### Adding DNS Provider Tests
When adding a new DNS provider, you **must** add contract tests:
```javascript
describe('NewProvider Provider', () => {
const NewProvider = require('../../models/dns_provider/newprovider');
test('should meet DNS provider contract', () => {
const mockCredentials = {api_key: 'mock-key'};
const instance = validateDnsProviderContract(NewProvider, mockCredentials);
assert.ok(instance);
});
test('should have valid method signatures', () => {
const instance = new NewProvider({api_key: 'mock'});
validateMethodSignatures(instance);
});
test('should validate key mapping', () => {
const instance = new NewProvider({api_key: 'mock'});
validateKeyMapping(instance);
});
test('should validate type checking', () => {
const instance = new NewProvider({api_key: 'mock'});
validateTypeChecking(instance);
});
});
```
See `test/integration/dns_provider.test.js` for examples.
## Code Style
### General Guidelines
- Use strict mode: `'use strict';`
- Use tabs for indentation
- Clear, descriptive variable names
- Comment complex logic
- No trailing whitespace
### File Organization
```javascript
'use strict';
// 1. Node.js built-ins
const fs = require('fs');
const path = require('path');
// 2. Third-party modules
const express = require('express');
const redis = require('redis');
// 3. Local modules
const {Host} = require('./models');
const middleware = require('./middleware/auth');
// 4. Code...
```
### Naming Conventions
- Classes: `PascalCase`
- Functions: `camelCase`
- Constants: `UPPER_SNAKE_CASE`
- Private methods: `__privateMethod` (double underscore prefix)
## Project Structure
Understanding the codebase:
```
nodejs/
├── models/ # Data models (Host, User, DNS providers)
├── routes/ # API route handlers
├── services/ # Background services (lookup, scheduler)
├── middleware/ # Express middleware
├── utils/ # Utility functions
├── public/ # Static web assets
├── views/ # EJS templates
└── test/ # Test suite
```
## Pull Request Process
### Before Submitting
1. **Run tests** - Ensure all tests pass
```bash
npm test
```
2. **Test locally** - Verify your changes work
```bash
npm run dev
```
3. **Update documentation** - Keep docs in sync with code changes
4. **Commit messages** - Use clear, descriptive messages
```
Add DNS provider for Route53
- Implement Route53 DNS API client
- Add contract tests for Route53
- Update documentation with Route53 setup
```
### Submitting a PR
1. **Fork the repository**
2. **Create a feature branch**
```bash
git checkout -b feature/my-new-feature
```
3. **Make your changes**
4. **Commit your changes**
```bash
git add .
git commit -m "Description of changes"
```
5. **Push to your fork**
```bash
git push origin feature/my-new-feature
```
6. **Open a Pull Request** on GitHub
### PR Requirements
- ✅ All tests must pass (CI/CD runs automatically)
- ✅ Tests run on Node.js 18.x, 20.x, and 22.x
- ✅ No merge conflicts with `master`
- ✅ Code follows project conventions
- ✅ New features include tests
- ✅ Documentation updated if needed
### CI/CD Process
When you open a PR:
1. GitHub Actions automatically runs tests
2. Tests execute on multiple Node.js versions
3. PR cannot be merged until all checks pass
4. Review from maintainers
5. Merge to master
## Adding Features
### Adding a DNS Provider
1. **Create provider file** in `models/dns_provider/yourprovider.js`
2. **Extend DnsApi base class**
```javascript
const {DnsApi} = require('./common');
class YourProvider extends DnsApi {
static _keyMap = {
api_key: {isRequired: true, type: 'string', isPrivate: true}
};
// Implement required methods
async listDomains() { }
async getRecords(domain, options) { }
async createRecord(domain, options) { }
async deleteRecords(domain, options) { }
}
```
3. **Add to provider list** in `models/dns_provider.js`
4. **Add contract tests** in `test/integration/dns_provider.test.js`
5. **Test your provider**
```bash
npm run test:integration
```
### Adding API Endpoints
1. **Add route** in appropriate file (`routes/`)
2. **Update API documentation** (`nodejs/api.md`)
3. **Test the endpoint** manually and add integration tests if needed
## Getting Help
- **Questions?** Open a [GitHub Discussion](https://github.com/theta42/proxy/discussions)
- **Bug reports** Use [GitHub Issues](https://github.com/theta42/proxy/issues)
- **Security issues** Email maintainers directly (see package.json)
## Code of Conduct
- Be respectful and inclusive
- Focus on constructive feedback
- Help others learn and grow
- Follow the project's technical direction
## License
By contributing, you agree that your contributions will be licensed under the MIT License.
---
[← Back to Home](index.html) | [View on GitHub](https://github.com/theta42/proxy)
+97
View File
@@ -0,0 +1,97 @@
---
layout: default
title: Home
---
# Proxy
A reverse proxy and HTTPS termination service using OpenResty/nginx with a management API and web GUI.
## Features
- **Automated HTTPS/SSL** - Let's Encrypt integration with HTTP-01 and DNS-01 challenges
- **Wildcard SSL Certificates** - Support for wildcard domains with automatic renewal
- **Multiple DNS Providers** - CloudFlare, DigitalOcean, PorkBun integrations
- **Advanced Routing** - Sophisticated wildcard domain matching (*, **)
- **RESTful API** - Full programmatic control
- **Web Interface** - User-friendly management GUI
- **High Performance** - Unix socket-based host lookup for minimal latency
## Quick Start
### Automated Installation
For modern Debian-based systems (Ubuntu 20.04+, Debian 11+):
```bash
wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash
```
### Requirements
- Node.js 18+ (tested with 18.x, 20.x, 22.x)
- OpenResty (nginx with Lua support)
- Redis
- Linux system with root access
## Documentation
- [Installation Guide](installation.html) - Detailed setup instructions
- [API Reference](api.html) - Complete API documentation
- [Architecture](architecture.html) - System design and components
- [Contributing](contributing.html) - Development and testing guide
## Use Cases
**Development Teams**
- Host multiple projects on a single server with unique domains
- Automatic SSL for all development sites
- Easy configuration via API or web UI
**Production Deployments**
- High-performance reverse proxy for microservices
- Centralized SSL certificate management
- Dynamic routing without nginx reloads
**Personal Projects**
- Self-hosted services with automatic HTTPS
- Wildcard certificates for unlimited subdomains
- Simple management interface
## Architecture
```
┌─────────────┐
│ Client │
└──────┬──────┘
│ HTTPS
┌─────────────────────┐
│ OpenResty/Nginx │
│ - SSL Termination │
│ - Host Routing │
└──────┬──────────────┘
│ Unix Socket
┌─────────────────────┐ ┌─────────────┐
│ Node.js API │◄────►│ Redis │
│ - Management │ │ - Storage │
│ - SSL Orchestration│ │ - Cache │
└──────┬──────────────┘ └─────────────┘
┌─────────────────────┐
│ Backend Services │
│ - Your Apps │
└─────────────────────┘
```
## Community
- [GitHub Repository](https://github.com/theta42/proxy)
- [Issue Tracker](https://github.com/theta42/proxy/issues)
- [Pull Requests](https://github.com/theta42/proxy/pulls)
## License
MIT License - See [LICENSE](https://github.com/theta42/proxy/blob/master/LICENSE) for details.
+221
View File
@@ -0,0 +1,221 @@
---
layout: default
title: Installation
---
# Installation Guide
[← Back to Home](index.html)
## Quick Install (Recommended)
For modern Debian-based systems (Ubuntu 20.04+, Debian 11+):
```bash
wget -O - https://raw.githubusercontent.com/theta42/proxy/master/ops/install.sh | sudo bash
```
This automated installer will:
- Install Node.js 20.x
- Install OpenResty and required dependencies
- Install and configure Redis
- Set up SSL fallback certificates
- Install Lua dependencies
- Clone and install the proxy application
- Configure systemd service
- Start the proxy service
## Manual Installation
### System Requirements
- Modern Linux distribution (Ubuntu 20.04+, Debian 11+, or equivalent)
- Root access
- Inbound internet access for Let's Encrypt validation
- Minimum 1GB RAM, 10GB disk space
### Step 1: Install Dependencies
**Ubuntu/Debian:**
```bash
apt install libpam0g-dev build-essential redis-server luarocks -y
```
### Step 2: Install Node.js 20.x
```bash
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \
sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
NODE_MAJOR=20
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | \
sudo tee /etc/apt/sources.list.d/nodesource.list
apt update && apt install nodejs -y
```
Verify installation:
```bash
node --version # Should show v20.x.x
npm --version
```
### Step 3: Install OpenResty
```bash
wget -O - https://openresty.org/package/pubkey.gpg | \
sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | \
sudo tee /etc/apt/sources.list.d/openresty.list
apt update && apt install openresty -y
```
### Step 4: Install Lua Dependencies
```bash
luarocks install lua-resty-auto-ssl
luarocks install luasocket
```
### Step 5: SSL Configuration
Create fallback SSL certificates:
```bash
mkdir -p /etc/ssl/
openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 \
-subj '/CN=sni-support-required-for-valid-ssl' \
-keyout /etc/ssl/resty-auto-ssl-fallback.key \
-out /etc/ssl/resty-auto-ssl-fallback.crt
```
### Step 6: Configure OpenResty
Clone the repository and copy configuration files:
```bash
cd /var/www
git clone https://github.com/theta42/proxy.git
cd proxy
# Copy nginx configs
mkdir -p /etc/openresty/sites-enabled/
cp ops/nginx_conf/nginx.conf /etc/openresty/nginx.conf
cp ops/nginx_conf/autossl.conf /etc/openresty/autossl.conf
cp ops/nginx_conf/proxy.conf /etc/openresty/sites-enabled/000-proxy
cp ops/nginx_conf/targetinfo.lua /usr/local/openresty/lualib/targetinfo.lua
```
### Step 7: Install Application
```bash
cd /var/www/proxy/nodejs
npm install
```
### Step 8: Configure Systemd Service
```bash
cp /var/www/proxy/ops/proxy.service /etc/systemd/system/proxy.service
systemctl daemon-reload
systemctl enable proxy.service
systemctl start proxy.service
```
Verify service is running:
```bash
systemctl status proxy.service
```
### Step 9: Initial Setup
The proxy API will be available on port 3000 by default. You'll need to:
1. Create your first user account
2. Configure DNS providers (for wildcard SSL)
3. Add your first host
See the [API Reference](api.html) for details.
## Configuration
### Environment Variables
- `NODE_ENV` - Set to `production` for production deployments
- `NODE_PORT` - Override default port (default: 3000)
### Redis Configuration
The proxy uses Redis with the prefix `proxy_`. To change this, edit `nodejs/conf/base.js`:
```javascript
redis: {
prefix: 'proxy_'
}
```
### OpenResty Configuration
Key configuration files in `/etc/openresty/`:
- `nginx.conf` - Main nginx configuration
- `autossl.conf` - Let's Encrypt HTTP-01 challenge handler
- `sites-enabled/000-proxy` - Proxy server configuration
### Unix Socket
The proxy communicates with OpenResty via Unix socket at:
```
/var/run/proxy_lookup.socket
```
This path is configurable in `nodejs/conf/base.js`.
## Troubleshooting
### Service won't start
Check logs:
```bash
journalctl -u proxy.service -f
```
Common issues:
- Port 3000 already in use
- Redis not running: `systemctl status redis-server`
- Permission issues: Service must run as root for user management
### SSL certificates not working
Check OpenResty logs:
```bash
tail -f /var/log/nginx/error.log
```
Common issues:
- Firewall blocking ports 80/443
- DNS not pointing to server
- Let's Encrypt rate limits exceeded
### Host lookup not working
Check Unix socket:
```bash
ls -la /var/run/proxy_lookup.socket
# Should show srwxrwxrwx (socket permissions)
```
Test lookup:
```bash
echo '{"domain":"example.com"}' | nc -U /var/run/proxy_lookup.socket
```
## Next Steps
- [Configure DNS Providers](api.html#dns-providers) for wildcard SSL
- [Add your first host](api.html#hosts)
- [Set up the web interface](index.html)
[← Back to Home](index.html)
+479 -69
View File
@@ -1,132 +1,542 @@
## get host info
# API Documentation
**GET** `/api/hosts<HOST>`
All API endpoints require authentication via the `auth-token` header unless otherwise noted.
Base URL: `https://your-proxy-host.com/api`
---
## Authentication
### Login
**POST** `/api/auth/login`
Authenticate a user and receive an auth token.
```bash
curl -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" https://proxy-host.com/api/hostsmine.com
curl -H "Content-Type: application/json" \
-X POST \
-d '{"username": "myuser", "password": "mypassword"}' \
https://proxy-host.com/api/auth/login
```
* 200 {"host":"yours.com","results":{"ip":"127.0.0.1:4000","updated":"1518595297563","username":"test10","forceSSL": false, "targetSSL": true, "targetPort": "443"}}
* 404 {"name": "HostNotFound", "message": "Host does not exists"}
**Responses:**
- `200` `{"login": true, "token": "027d3964-7d81-4462-a6f9-2c1f9b40b4be", "message": "myuser logged in!"}`
- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}`
### Logout
## view all hosts
**ALL** `/api/auth/logout`
**GET** `/api/hosts`
Invalidate the current auth token.
```bash
curl -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" https://proxy-host.com/api/hosts
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/auth/logout
```
* 200 {"hosts":["mine.com","mine2.com"]}
**Responses:**
- `200` `{"message": "Bye"}`
---
## Add host
## Users
**POST** `/api/hosts`
All user endpoints require authentication.
Params
* **host** -- Required, The domain name for the new record.
* **ip** -- Required, The target IP or FQDN for the record.
* **targetSSL** -- If the remote IP target is SSL. Default is false and this is
not recommended.
* **targetPort** -- Required, TCP port for the remote server. Unless you know
otherwise, 80 for targetSSL false and 443 for true.
* **forceSSL** -- If requests should be forced to use SSL from the client to
the proxy. The default is false and this is HIGHLY recommended.
* **
### List Users
**GET** `/api/user`
Get list of all users.
```bash
curl -H "Content-Type: application/json" -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" -X POST -d '{"host": "test.vm42.com", "ip": "192.168.1.21", "targetSSL": false, "targetPort": "443", "forceSSL": true} https://proxy-host.com/api/hosts
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/user
```
* 200 {"message":"Host yours.com added."}
* 409 {"name":"HostNameUsed", "message":"Host already exists"}
* 422 {"name":"ObjectValidateError","message":[{"key":"ip","message":"ip is required."}]} Missing or incorrect keys/values. Returns a list with a message per key error.
**Query Parameters:**
- `detail` - Include full user details (optional)
## Edit
**Responses:**
- `200` `{"results": ["user1", "user2"]}`
- `200` `{"results": [{"username": "user1", ...}, ...]}` (with `?detail=true`)
**PUT** `/api/hosts<host>`
### Get Current User
Takes the same params as add, but none are required
**GET** `/api/user/me`
curl -H "Content-Type: application/json" -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" -X POST -d '{"host": "test.vm42.com", "ip": "192.168.1.21", "targetSSL": false, "targetPort": "443", "forceSSL": true} https://proxy-host.com/api/hosts
* 200 {"message":"Host yours.com updated."}
* 404 {"name": "HostNotFound", "message": "Host does not exists"}
* 409 {"name":"HostNameUsed", "message":"Host already exists"}
* 422 {"name":"ObjectValidateError","message":[{"key":"ip","message":"ip is required."}]} Missing or incorrect keys/values. Returns a list with a message per key error.
## delete host
**DELETE** /`api/hosts/<host>`
Get information about the currently authenticated user.
```bash
curl -H "Content-Type: application/json" -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" -X DELETE https://proxy-host.com/api/hosts
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/user/me
```
* 200 {"message":"Host yours.com deleted"}
* 404 {"name": "HostNotFound", "message": "Host does not exists"}
**Responses:**
- `200` `{"username": "myuser"}`
### Create User
## create invite token
**POST** `/api/user`
**post** `/users/invite`
Create a new user.
```bash
curl -H "Content-Type: application/json" -H "auth-token: 0b06eb2e-4ca4-4881-9a0f-b8df55431cd1" -X POST https://proxy-host.com/users/invite
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"username": "newuser", "password": "newpassword"}' \
https://proxy-host.com/api/user
```
* 200 {"token":"5caf94d2-2c91-4010-8df7-968d10802b9d"}
**Responses:**
- `200` User created successfully
- `409` Username already exists
- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error
### Delete User
## sing up
**DELETE** `/api/user/:username`
**post** `/auth/invite/<INVITE TOKEN>`
Delete a user account.
```bash
curl -H "Content-Type: application/json" -X POST -d "{\"username\": \"test9\", \"password\": \"palm7\"}" https://proxy-host.com/auth/invite/b33d8819-ec64-4cf4-a6ec-77562d738fa4
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/user/olduser
```
* 200 {"user":"test9","token":"af662d8b-3d44-4110-8ad9-047dc752d97f"}
* 400 {"message":"Missing fields"}
* 401 {"message":"Token not valid"}
* 409 {"message":"username taken"}
**Responses:**
- `200` `{"username": "olduser", "results": ...}`
- `404` User not found
### Change Password (Self)
## login
**PUT** `/api/user/password`
**post** `/auth/login`
Change the password for the currently authenticated user.
```bash
curl -H "Content-Type: application/json" -X POST -d '{"username": "test8", "password": "mypassword"}' https://proxy-host.com/auth/login
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"password": "newpassword"}' \
https://proxy-host.com/api/user/password
```
* 200 {"login":true,"token":"027d3964-7d81-4462-a6f9-2c1f9b40b4be"}
* 401 {"login":false}
**Responses:**
- `200` `{"results": ...}` Password changed successfully
### Change Password (Other User)
## verify SSH key
**PUT** `/api/user/password/:username`
**post** `/auth/verifykey`
Change the password for another user (admin function).
```bash
curl -H "Content-Type: application/json" -X POST -d "{\"key\":\"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDM9vboz5YGgESsrR2e4JOeP2qtmQo2S8BjI+Y/VxPQ6WbNFzAkXxDniHcnPCrhkeX36SKINvMjWnt4XOK2S+X+1tCoXJzqtcKKyK0gx8ijBxcWVPxsMWjMYTGSVSKiKnt6CyQzrbVGJMh3iAQ8Yv1JwH+6SAtMgT8it7iLyntNFJCesh4I/znEG58A5VBbdUle1Ztz9afjj1CZns17jk7KPm9ig5DmuvdvnMEfhFjfKv1Rp6S5nxacMoTP4tJNSEUh55IicoWk94ii5GwUVLYgyMmzdlA32TqVLFpU2yAvdA9WSnBaI/ZyktlfI7YAmK2wFBsagr9Pq1TcUAY6rZ/GTMjDxExgdYn/FxlufcuqeNJsJXs2A+0xDS/9mv/yGQzNZrL8DrVhY2OKKLoH4Q7enDbhSgEFmJUJMqPxuPEgLEvKfzcURSvIwRj1iCEw6S4dhdaLJl2RRBb1ZWBQbE5ogIbvAl7GFJUAhj3pqYJnd30VENv1MkK+IoCS7EEP0caqL9RNAId0Plud7q2XElHqzkYUE+z+Q/LvGgclXK1ZmZejNaMnV53wfhAevfwVyNGK9i5gbwc1P2lplIa5laXCcVWezqELEkTpdjp4AeKmMuCr8rY8EnLKIcKWEOsX5UumztCow6e1E55v3VeHvRZLpw4DZP7EE0Q8B/jPFWqbCw== wmantly@gmail.com\"}" https://proxy-host.com/auth/verifykey
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"password": "newpassword"}' \
https://proxy-host.com/api/user/password/otheruser
```
* 200 {"info":"4096 SHA256:dfdCYzt0atMBXVZTJzUxsu99IjXXFXpocSox5q+jOs8 wmantly@gmail.com (RSA)\n"}
* 400 {"message":"Key is not a public key file!"}
**Responses:**
- `200` `{"results": ...}` Password changed successfully
- `404` User not found
### Create Invite Token
## add ssh key to current user
**POST** `/api/user/invite`
**post** `/users/key`
Create an invitation token for new user registration.
```bash
curl -H "Content-Type: application/json" -H "auth-token: 8eff4f16-086d-40fd-acbd-7634b9a36117" -X POST -d "{\"key\": \"ssh-rsa AAAAB3NzaC1yc2EAAjWnt4XOK2S+X+1tCoXJzqtcKKyK0gx8ijBxcWVPxsMWjMYTGSVSKiKnt6CyQzrbVGJMh3iAQ8Yv1JwH+6SAtMgT8it7iLyntNFJCesh4I/znEG58A5VBbdUle1Ztz9afjj1CZns17jk7KPm9ig5DmuvdvnMEfhFjfKv1Rp6S5nxacMoTP4tJNSEUh55IicoWk94ii5GwUVLYgyMmzdlA32TqVLFpU2yAvdA9WSnBaI/ZyktlfI7YAmK2wFBsagr9Pq1TcUAY6rZ/GTMjDxExgdYn/FxlufcuqeNJsJXs2A+0xDS/9mv/yGQzNZrL8DrVhY2OKKLoH4Q7enDbhSgEFmJUJMqPxuPEgLEvKfzcURSvIwRj1iCEw6S4dhdaLJl2RRBb1ZWBQbE5ogIbvAl7GFJUAhj3pqYJnd30VENv1MkK+IoCS7EEP0caqL9RNAId0Plud7q2XElHqzkYUE+z+Q/LvGgclXK1ZmZejNaMnV53wfhAevfwVyNGK9i5gbwc1P2lplIa5laXCcVWezqELEkTpdjp4AeKmMuCr8rY8EnLKIcKWEOsX5UumztCow6e1E55v3VeHvRZLpw4DZP7EE0Q8B/jPFWqbCw== wmantly@gmail.co\"}" https://proxy-host.com/users/key
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/user/invite
```
* 200 {"message":true}
* 400 {"message":"Bad SSH key"}
**Responses:**
- `200` `{"token": "5caf94d2-2c91-4010-8df7-968d10802b9d"}`
### Add SSH Key
**POST** `/api/user/key`
Add an SSH public key to the current user's account.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"key": "ssh-rsa AAAAB3..."}' \
https://proxy-host.com/api/user/key
```
**Responses:**
- `200` `{"message": true}` Key added successfully
- `400` `{"message": "Bad SSH key"}` Invalid key format
---
## Hosts
Manage proxy host configurations.
### List Hosts
**GET** `/api/host`
Get list of all configured hosts.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host
```
**Query Parameters:**
- `detail` - Include full host details (optional)
**Responses:**
- `200` `{"results": ["example.com", "*.wildcard.com"]}`
- `200` `{"results": [{"host": "example.com", "ip": "192.168.1.10", ...}, ...]}` (with `?detail=true`)
### Get Host
**GET** `/api/host/:host`
Get configuration for a specific host.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"item": "example.com", "results": {"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, ...}}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
### Lookup Host
**GET** `/api/host/lookup/:domain`
Test the host lookup algorithm (supports wildcard matching).
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/lookup/sub.example.com
```
**Responses:**
- `200` `{"string": "sub.example.com", "results": {"host": "*.example.com", ...}}`
- `200` `{"string": "sub.example.com", "results": null}` (no match)
### Get Lookup Tree
**GET** `/api/host/lookupobj`
Get the internal lookup tree structure (for debugging).
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/host/lookupobj
```
**Responses:**
- `200` `{"results": {"com": {"example": {...}}}}`
### Create Host
**POST** `/api/host`
Add a new host configuration.
**Parameters:**
- `host` (required) - Domain name (e.g., `example.com`, `*.example.com`)
- `ip` (required) - Target IP address or FQDN
- `targetPort` (required) - Target port number (1-65535)
- `forcessl` (optional) - Force HTTPS redirect (default: true)
- `targetssl` (optional) - Use HTTPS to backend (default: false)
- `challengeType` (optional) - For wildcards: `DNS-01-wildcard` or `wildcardChild`
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"host": "example.com", "ip": "192.168.1.10", "targetPort": 8080, "forcessl": true, "targetssl": false}' \
https://proxy-host.com/api/host
```
**Responses:**
- `200` `{"message": "\"example.com\" added.", "host": "example.com", ...}`
- `409` `{"name": "HostNameUsed", "message": "Host already exists"}`
- `422` `{"name": "ObjectValidateError", "message": ...}` Validation error
### Update Host
**PUT** `/api/host/:host`
Update an existing host configuration.
**Parameters:** Same as Create Host (all optional)
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"ip": "192.168.1.20", "targetPort": 9000}' \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"message": "\"example.com\" updated.", ...}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
- `422` Validation error
### Delete Host
**DELETE** `/api/host/:host`
Remove a host configuration.
```bash
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/host/example.com
```
**Responses:**
- `200` `{"message": "example.com deleted", ...}`
- `404` `{"name": "HostNotFound", "message": "Host does not exists"}`
### Renew Wildcard Certificate
**PUT** `/api/host/:host/renew`
Manually trigger wildcard certificate renewal.
```bash
curl -H "auth-token: your-token-here" \
-X PUT \
https://proxy-host.com/api/host/*.example.com/renew
```
**Responses:**
- `200` `{"message": "Requesting wildcard cert for *.example.com"}`
- `404` Host not found
---
## DNS Providers
Manage DNS provider integrations for wildcard SSL certificates.
### List DNS Providers
**GET** `/api/dns`
Get list of configured DNS providers.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns
```
**Query Parameters:**
- `detail` - Include full provider details (optional)
**Responses:**
- `200` `{"results": ["provider-id-1", "provider-id-2"]}`
### List Available Provider Types
**OPTIONS** `/api/dns`
Get list of supported DNS provider types and their configuration requirements.
```bash
curl -H "auth-token: your-token-here" \
-X OPTIONS \
https://proxy-host.com/api/dns
```
**Responses:**
- `200` `{"results": [{"name": "CloudFlare", "fields": {...}}, {"name": "DigitalOcean", ...}, ...]}`
### Create DNS Provider
**POST** `/api/dns`
Configure a new DNS provider.
**CloudFlare:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My CloudFlare", "dnsProvider": "Cloudflare", "token": "your-api-token"}' \
https://proxy-host.com/api/dns
```
**DigitalOcean:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My DO", "dnsProvider": "DigitalOcean", "token": "your-api-token"}' \
https://proxy-host.com/api/dns
```
**PorkBun:**
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X POST \
-d '{"name": "My PorkBun", "dnsProvider": "PorkBun", "apiKey": "pk_xxx", "secretApiKey": "sk_xxx"}' \
https://proxy-host.com/api/dns
```
**Responses:**
- `200` `{"message": "\"provider-id\" added.", ...}`
- `422` Validation error or invalid API credentials
### Get DNS Provider
**GET** `/api/dns/:id`
Get a specific DNS provider configuration.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"item": "provider-id", "results": {...}}`
- `404` Provider not found
### Update DNS Provider
**PUT** `/api/dns/:id`
Update DNS provider configuration.
```bash
curl -H "Content-Type: application/json" \
-H "auth-token: your-token-here" \
-X PUT \
-d '{"name": "Updated Name"}' \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"message": "\"provider-id\" updated.", ...}`
- `404` Provider not found
### Delete DNS Provider
**DELETE** `/api/dns/:id`
Remove a DNS provider and all associated domains.
```bash
curl -H "auth-token: your-token-here" \
-X DELETE \
https://proxy-host.com/api/dns/provider-id
```
**Responses:**
- `200` `{"message": "provider-id deleted", ...}`
- `404` Provider not found
### List Domains
**GET** `/api/dns/domain`
List all domains from all configured providers.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/domain
```
**Query Parameters:**
- `detail` - Include full domain details (optional)
**Responses:**
- `200` `{"results": ["example.com", "test.com"]}`
### Get Domain
**GET** `/api/dns/domain/:domain`
Get details for a specific domain.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/dns/domain/example.com
```
**Responses:**
- `200` `{"results": [{"domain": "example.com", "zoneId": "...", ...}]}`
- `404` Domain not found
### Refresh Domains
**POST** `/api/dns/domain/refresh/:providerId`
Refresh the domain list from a DNS provider's API.
```bash
curl -H "auth-token: your-token-here" \
-X POST \
https://proxy-host.com/api/dns/domain/refresh/provider-id
```
**Responses:**
- `200` `{"results": ...}` Updated domain list
- `404` Provider not found
---
## Certificates
Retrieve SSL certificate information.
### Get Certificate
**GET** `/api/cert/:host`
Get the SSL certificate for a host.
```bash
curl -H "auth-token: your-token-here" \
https://proxy-host.com/api/cert/example.com
```
**Responses:**
- `200` Certificate data including `cert_pem`, `fullchain_pem`, `privkey_pem`, expiry information
- `404` Certificate not found
---
## Error Responses
All endpoints may return the following error responses:
- `401` `{"name": "LoginFailed", "message": "Invalid Credentials, login failed."}` - Authentication required or invalid
- `404` `{"name": "NotFound", "message": "..."}` - Resource not found
- `422` `{"name": "ObjectValidateError", "message": [...], "keys": [...]}` - Validation errors
- `500` Internal server error
## Notes
- All timestamps are in milliseconds since epoch
- The `auth-token` header is required for all authenticated endpoints
- Host names support wildcards: `*` (single level) and `**` (multi-level)
- DNS providers are validated on creation - invalid API credentials will be rejected
- Wildcard certificates are automatically renewed 30 days before expiration
+9
View File
@@ -19,6 +19,15 @@ const middleware = require('./middleware/auth');
// Grab the projects PubSub
app.contoller = require('./controller');
/**
* Start background services
* These services run independently of the HTTP server:
* - host_lookup: Unix socket server for OpenResty host lookups
* - host_scheduler: Scheduled tasks for wildcard cert renewal
*/
require('./services/host_lookup');
require('./services/host_scheduler');
// Push pubsub over the socket and back.
app.onListen.push(function(){
app.io.use(middleware.authIO);
-49
View File
@@ -1,49 +0,0 @@
'use strict';
const Table = require('../models');
const {User, AuthToken} = Table.models;
class Auth{
static errors = {
login: function(){
let error = new Error('LoginFailed');
error.name = 'LoginFailed';
error.message = `Invalid Credentials, login failed.`;
error.status = 401;
return error;
}
}
static async login(data){
try{
let user = await User.login(data);
let token = await AuthToken.create({username: user.username});
return {user, token}
}catch(error){
console.log('login error', error);
throw this.errors.login();
}
}
static async checkToken(token){
try{
token = await AuthToken.get(token);
if(token && token.check()) return token;
throw this.errors.login();
}catch(error){
console.log('check error', error);
throw this.errors.login();
}
}
static async logout(data){
let token = await AuthToken.get(data);
await token.destroy();
}
}
module.exports = {Auth};
-33
View File
@@ -1,33 +0,0 @@
'use strict';
const {Host} = require('../models/host');
const {SocketServerJson} = require('../models/socket_server_json');
const conf = require('../conf');
const socket = new SocketServerJson({
socketFile: conf.socketFile,
onData: function(data, clientSocket) {
try{
console.log('socket lookup', data)
let parentHost = Host.lookUp(data['domain']);
console.log('socket found host', parentHost);
if(!parentHost) return clientSocket.write(JSON.stringify({}));
if(!parentHost.wildcard_parent){
parentHost.wildcard_parent = parentHost.host;
Host.addCache(data['domain'], parentHost);
}
for(const [key, value] of Object.entries(parentHost)) {
parentHost[key] = String(value);
};
clientSocket.write(JSON.stringify(parentHost));
}catch(error){
console.error('controler/hosts onData error', error)
}
},
onListen: function(){
console.log('Unix socket listening on', conf.socketFile)
}
});
-2
View File
@@ -1,8 +1,6 @@
'use strict';
module.exports = {
auth: require('./auth'),
ps: require('./pubsub'),
host: require('./host'),
}
-25
View File
@@ -1,25 +0,0 @@
#! /usr/bin/env node
//you must add the absolute path to hosts.js on line 4, or this file will NOT work
const hosts = require("./models/hosts.js")
const command = process.argv[2];
(async function(command){
if (command == "--list"){
console.log(await hosts.listAll())
process.exit(0)
}
})(command)
/*
if process.argv[2] == "--info"
if process.agrv[2] == "--add"
if process.argv[2] == "--remove"
else{
console.log("help text")
}
*/
+1 -1
View File
@@ -1,6 +1,6 @@
'use strict';
const {Auth} = require('../controller/auth');
const {Auth} = require('../models/auth');
async function auth(req, res, next){
try{
+96
View File
@@ -0,0 +1,96 @@
'use strict';
const Table = require('../models');
const {User, AuthToken} = Table.models;
/**
* Auth Model
*
* Handles authentication operations for the application.
* Manages user login, token validation, and logout processes.
*
* Dependencies:
* - User model: Validates user credentials
* - AuthToken model: Creates and manages authentication tokens
*
* All methods throw standardized login errors on failure to avoid
* leaking information about whether usernames exist or tokens are valid.
*/
class Auth{
/**
* Standardized error responses for authentication failures.
* Returns generic "Invalid Credentials" message for security.
*/
static errors = {
login: function(){
let error = new Error('LoginFailed');
error.name = 'LoginFailed';
error.message = `Invalid Credentials, login failed.`;
error.status = 401;
return error;
}
}
/**
* Authenticate user and create session token.
*
* @param {Object} data - Login credentials {username, password}
* @returns {Object} {user, token} - User object and auth token
* @throws {Error} Generic login error on any failure
*
* Flow:
* 1. Validate credentials via User.login()
* 2. Create new AuthToken for the user
* 3. Return both user data and token
*/
static async login(data){
try{
let user = await User.login(data);
let token = await AuthToken.create({username: user.username});
return {user, token}
}catch(error){
console.log('login error', error);
throw this.errors.login();
}
}
/**
* Validate an authentication token.
*
* @param {string} token - Token string to validate
* @returns {Object} Token object if valid
* @throws {Error} Generic login error if token invalid or expired
*
* Checks:
* 1. Token exists in database
* 2. Token has not expired (via token.check())
*/
static async checkToken(token){
try{
token = await AuthToken.get(token);
if(token && token.check()) return token;
throw this.errors.login();
}catch(error){
console.log('check error', error);
throw this.errors.login();
}
}
/**
* Destroy an authentication token (logout).
*
* @param {string} data - Token string to destroy
* @returns {void}
*
* Removes token from database, invalidating the session.
*/
static async logout(data){
let token = await AuthToken.get(data);
await token.destroy();
}
}
module.exports = {Auth};
+44 -10
View File
@@ -39,17 +39,12 @@ class Host extends Table{
static async addCache(host, parentOBJ){
try{
console.log('addCache host:', host, 'parentOBJ host', parentOBJ.host)
parentOBJ = await this.get(parentOBJ.host);
if(parentOBJ.is_cache){
console.log('addCache parentOBJ is chace, skipping')
return;
}
console.log('addCache, corrent parent?', parentOBJ.wildcard_parent || parentOBJ.host)
console.log('addCache, got parent', parentOBJ)
await this.create({
...parentOBJ,
host: host,
@@ -63,7 +58,7 @@ class Host extends Table{
parent: parentOBJ.host
});
}catch(error){
console.error('add cache error', {...parentOBJ, host, is_cache: true}, error)
console.error('add cache error', {...parentOBJ, host, is_cache: true}, error);
throw error;
}
}
@@ -81,18 +76,35 @@ class Host extends Table{
}catch(error){
console.error('bust cache error', error)
throw error;
}
}
static async create(data, ...args){
try{
if(data.is_wildcard) await this.validateWildcardCreate(data, args);
// Validate requested host is valid host and domain
if(data.challengeType === 'DNS-01-wildcard') await this.validateWildcardCreate(data, args);
// Validate requested host has a valid wildcard parent
if(data.challengeType === 'wildcardChild'){
let parentHost = await this.lookUp(data.host);
console.log('parentHost:', parentHost)
if(parentHost.is_wildcard){
data.wildcard_parent = parentHost.host;
}else{
throw new Error(`No parent wild card for ${data.host}`);
}
}
// Create the new host entry
let out = await super.create(data, ...args);
// Update the lookup table to reflect new host
await this.buildLookUpObj();
if(out.is_wildcard) out.createWildcardCert();
// Fire the request for the wild card cert
// This is "back ground" job, await is intentionally missing
if(out.challengeType === 'DNS-01-wildcard') out.createWildcardCert();
return out;
@@ -134,7 +146,8 @@ class Host extends Table{
type:'TXT',
name: `_acme-challenge${parts.sub ? `.${parts.sub}` : ''}`,
data: `${keyAuthorization}`
}
},
true // Force the record creation, even if the record exists
);
}catch(error){
console.log('model Host challengeCreateFn error:', error)
@@ -215,6 +228,26 @@ class Host extends Table{
}
}
async checkWildcardForRenew(){
try{
if(this.is_wildcard && Date.now() > this.wildcard_expires - (30 * 24 * 60 * 60 * 1000)){
this.createWildcardCert();
}
}catch(error){
throw error;
}
}
static async checkWildcardForRenew(){
try{
for(let host of await this.listDetail()){
host.createWildcardCert();
}
}catch(error){
throw error;
}
}
async update(...args){
try{
let out = await super.update(...args)
@@ -330,6 +363,7 @@ class Host extends Table{
// Check every 5ms to see if the look up tree is ready
while(!this.__lookUpIsReady) await new Promise(r => setTimeout(r, 5));
return true;
}
}
-88
View File
@@ -1,88 +0,0 @@
'use strict';
const net = require('net');
const fs = require('fs');
const {CallbackQueue} = require('../utils/callback_queue')
class SocketServerJson {
constructor(args){
this.socketFile = args.socketFile;
this.onData = new CallbackQueue(args.onData, this);
this.onListen = new CallbackQueue(args.onListen, this);
this.onError = new CallbackQueue(args.onError);
this.onCLientNew = new CallbackQueue(args.onCLientNew);
this.onCLientClose = new CallbackQueue(args.onCLientClose);
this.onCLientError = new CallbackQueue(args.onCLientClose);
this.onListen.push(function(){
fs.chmodSync(args.socketFile, '777');
})
this.listen();
}
__resetSocketFile(callback){
let instance = this;
fs.stat(this.socketFile, function (err, stats) {
if (stats) {
fs.unlink(instance.socketFile, function(err){
if(err){
// This should never happen.
console.error(err);
}
callback(...arguments)
});
}else{
callback()
}
});
}
__setUpServer(){
let instance = this;
this.socket = net.createServer();
this.socket.on('connection', function(clientSocket){
let buffer = '';
clientSocket.on('data', function(data){
buffer += data.toString();
try{
instance.onData.call(JSON.parse(data), clientSocket)
buffer = ''
// clientSocket.write(JSON.stringify(Host.lookUp(buffer)|| {host: 'none'}));
}catch(error){
;
}
});
clientSocket.on('close', instance.onCLientClose.call.bind(instance.onCLientClose));
clientSocket.on('error', instance.onCLientError.call.bind(instance.onCLientError));
});
this.socket.on('error', this.onError.call.bind(this.onError))
this.socket.on('listening', this.onListen.call.bind(this.onListen));
}
listen(){
let instance = this;
this.__setUpServer();
this.__resetSocketFile(function(){
instance.socket.listen(instance.socketFile);
});
}
};
module.exports = {SocketServerJson};
+822 -531
View File
File diff suppressed because it is too large Load Diff
+10 -1
View File
@@ -9,7 +9,15 @@
}
],
"scripts": {
"start": "node ./bin/www"
"start": "node ./bin/www",
"dev": "npx nodemon --ignore public/ ./bin/www",
"test": "node --test test/unit/callback_queue.test.js test/unit/host_lookup.test.js test/unit/unix_socket.test.js test/integration/dns_provider.test.js",
"test:unit": "node --test test/unit/callback_queue.test.js test/unit/host_lookup.test.js test/unit/unix_socket.test.js",
"test:integration": "node --test test/integration/dns_provider.test.js",
"test:watch": "node --test --watch test/unit/callback_queue.test.js test/unit/host_lookup.test.js test/unit/unix_socket.test.js test/integration/dns_provider.test.js"
},
"engines": {
"node": ">=18.0.0"
},
"dependencies": {
"@fortawesome/fontawesome-free": "^6.4.2",
@@ -21,6 +29,7 @@
"ejs": "^3.1.10",
"express": "^4.19.2",
"extend": "^3.0.2",
"jq-repeat": "^2.0.0",
"jquery": "^3.7.1",
"ldapts": "^2.12.0",
"linux-sys-user": "^1.1.8",
+4
View File
@@ -12,3 +12,7 @@ nav.navbar{
font-weight: bold;
}
.actionMessage{
position: fixed;
}
+23 -16
View File
@@ -304,31 +304,38 @@ app.util = (function(app){
setTimeout(callback,10)
}
$.fn.serializeObject = function(){
var
arr = $(this).serializeArray(),
obj = {};
$.fn.serializeObject = function() {
var obj = {};
for(var i = 0; i < arr.length; i++){
if(obj[arr[i].name] === undefined) {
if(!arr[i].value) continue;
obj[arr[i].name] = arr[i].value;
let type = $(this).parent().find(`[name="${arr[i].name}"]`).attr('type');
if(['number', 'range'].includes(type)){
obj[arr[i].name] = Number(arr[i].value);
// Get the form values and work over them
for (let {name, value} of $(this).serializeArray()) {
console.log(name, value)
if (obj[name] === undefined) {
if (!value
&& !$(this).parent().find(`[name="${name}"]`).attr('value')
){
continue;
}
if(['radio'].includes(type) && ['true', 'false'].includes(arr[i].value)){
obj[arr[i].name] = arr[i].value == 'true' ? true : false;
obj[name] = value;
let type = $(this).parent().find(`[name="${name}"]`).attr('type');
if (['number', 'range'].includes(type)) {
obj[name] = Number(value);
}
if (['radio'].includes(type) && ['true', 'false'].includes(value)) {
obj[name] = value == 'true' ? true : false;
}
} else {
if(!(obj[arr[i].name] instanceof Array)) {
obj[arr[i].name] = [obj[arr[i].name]];
if (!(obj[name] instanceof Array)) {
obj[name] = [obj[name]];
}
obj[arr[i].name].push(arr[i].value);
obj[name].push(value);
}
}
return obj;
};
-394
View File
@@ -1,394 +0,0 @@
/*
Author William Mantly Jr <wmantly@gmail.com>
https://github.com/wmantly/jq-repeat
MIT license
*/
(function($, Mustache){
'use strict';
var scope = {};
$.scope = new Proxy(scope, {
get(obj, prop){
if(!obj[prop]){
scope[prop] = [];
}
return Reflect.get(...arguments);
},
set(obj, prop, value) {
return Reflect.set(...arguments);
},
});
var make = function(element, template){
var result = [];
result.splice = function(inputValue, ...args){
//splice does all the heavy lifting by interacting with the DOM elements.
var toProto = [...args]
var index;
//if a string is submitted as the index, try to match it to index number
if(typeof arguments[0] === 'string'){
index = this.indexOf( arguments[0] );//set where to start
if (index === -1) {
return [];
}
}else{
index = arguments[0]; //set where to start
}
toProto.unshift(index)
var howMany = arguments[1]; //sets the amount of fields to remove
var args = Array.prototype.slice.call( arguments ); // coverts arguments into array
var toAdd = args.slice(2); // only keeps fields to add to array
// if the starting point is higher then the total index count, start at the end
if( index > this.length ) {
index = this.length;
}
// if the starting point is negative, start form the end of the array, minus the start point
if( index < 0 ) {
index = this.length - Math.abs( index );
}
// if there are things to add, figure out the how many new indexes we need
if( !howMany && howMany !== 0 ) {
howMany = this.length - index;
}
//not sure why i put this here... but it does matter!
if( howMany > this.length - index ) {
howMany = this.length - index;
}
//figure out how many positions we need to shift the current elements
var shift = toAdd.length - howMany;
// figure out how big the new array will be
// var newLength = this.length + shift;
//removes fields from array based on howMany needs to be removed
for( var i = index; i < +index+howMany; i++ ) {
this.__take(this[index].__jq_$el, this[index], this);
// this.__take.apply( $( '.jq-repeat-'+ this.__jqRepeatId +'[jq-repeat-index="'+ ( i + index ) +'"]' ) );
}
//re-factor element index's
for(var i = 0; i < this.length; i++){
if( i >= index){
this[i].__jq_$el.attr( 'jq-repeat-index', i+shift );
}
}
//if there are fields to add to the array, add them
if( toAdd.length > 0 ){
//$.each( toAdd, function( key, value ){
for(var I = 0; I < toAdd.length; I++){
//figure out new elements index
var key = I + index;
// apply values to template
var render = Mustache.render(this.__jqTemplate, this.__buildData(i, toAdd[I]));
//set call name and index keys to DOM element
var $render = $( render ).addClass( 'jq-repeat-'+ this.__jqRepeatId ).attr( 'jq-repeat-index', key );
//if add new elements in proper stop, or after the place holder.
if( key === 0 ){
this.$this.after( $render );
}else{
$( '.jq-repeat-'+ this.__jqRepeatId +'[jq-repeat-index="' + ( key -1 ) + '"]' ).after( $render );
}
Object.defineProperty( toAdd[I], "__jq_$el", {
value: $render,
writable: true,
enumerable: false,
configurable: true
} );
//animate element
this.__put($render, toAdd[I], this);
}
}
//set and return new array
return Array.prototype.splice.apply(this, toProto);
};
result.push = function(){
//add one or more objects to the array
//set the index value, if none is set make it zero
var index = this.length || 0;
//loop each passed object and pass it to slice
for (var i = 0 ; i < arguments.length; ++i) {
this.splice( ( index + i ), 0, arguments[i] );
}
//return new array length
return this.length;
};
result.pop = function(){
//remove and return array element
return this.splice( -1, 1 )[0];
};
result.reverse = function() {
let hold = [];
for(let item of this){
hold.push(item.__jq_$el.html())
}
for(let idx in hold.reverse()){
this[idx].__jq_$el.html(hold[idx]);
}
Array.prototype.reverse.apply( this );
return this;
};
result.remove = function(key, value){
let index = this.indexOf(key, value)
if(index === -1) return;
this.splice(index, 1)
}
result.shift = function() {
return this.splice( 0, 1 )[0];
};
result.unshift = function(data){
return this.splice(0,0, data)
}
result.loop = function(){
var temp = this[0];
this.splice( 0,1 );
this.push( temp );
return temp;
};
result.loopUp = function(){
var temp = this[this.length-1];
this.splice( -1, 1 );
this.splice( 0, 0, temp );
return temp;
};
result.indexOf = function( key, value ){
if(typeof key === 'number') return key;
if( typeof value !== 'string' ){
value = arguments[0];
key = this.__index;
}
for ( var index = 0; index < this.length; ++index ) {
if( this[index][key] === value ){
return index;
}
}
return -1;
};
result.update = function(key, value, data){
//set variables using sting for index
// If update is called with no index/key, assume its the 0
if(typeof key === 'object'){
if(this[0]){
return this.update(0, key);
}
return this.splice(0, 1, key);
}
if(typeof value !== 'string'){
data = arguments[1];
if(typeof key !== 'number'){
value = arguments[0];
key = this.__index;
}
}
var index = this.indexOf( key, value );
if(index === -1) {
return [];
}
this[index] = $.extend( true, this[index], data );
var $render = $(Mustache.render(this.__jqTemplate, this.__buildData(index, this[index])));
$render.attr('jq-repeat-index', index);
this.__putUpdate(this[index].__jq_$el, $render, this[index], this);
this[index].__jq_$el = $render;
};
result.getByKey = function(key, value){
return this[this.indexOf(key, value)];
}
// User definable helper methods
result.__put = function($el, item, list){
$el.show();
};
result.__take = function($el, item, list){
$el.remove();
};
result.__putUpdate = function($el, $render, item, list){
$el.replaceWith($render);
$el.show();
};
result.__parseData = function(data){
return data;
}
// internal helper methods
result.__buildData = function(index, data){
return {
...this.__parseData(data),
nestedTemplates: this.__parseNestedTemplates(index, data),
_parent: this._parentData,
};
};
result.__parseNestedTemplates = function(index, data){
let templates = []
let tempData = {
...data,
_parent: data,
};
for(let idx in this.nestedTemplates){
let $el = $(`${this.nestedTemplates[idx]}`);
$el.attr('jq-repeat', Mustache.render($el.attr('jq-repeat'), tempData));
$el.attr('jq-repeat-index', Mustache.render($el.attr('jq-repeat-index'), tempData));
$el.attr('jq-repeat-parent', this.__jqRepeatId);
$el.attr('jq-repeat-parent-index', index);
templates[idx] = $el[0].outerHTML;
}
return templates;
}
for(let prop of ['put', 'take', 'putUpdate', 'parseData']){
Object.defineProperty(result, prop, {
enumerable: false,
get(){
return this[`__${prop}`]
},
set(value) {
this[`__${prop}`] = value;
},
});
}
var $this = $(element);
result.nestedTemplates = [];
result.__jqRepeatId = $this.attr( 'jq-repeat' );
$this.removeAttr('jq-repeat');
result.__index = $this.attr('jq-repeat-index');
if($this.attr('jq-repeat-parent')){
result._parentData = $.scope[$this.attr('jq-repeat-parent')][$this.attr('jq-repeat-parent-index')]
result.__jqParent = $this.attr('jq-repeat-parent');
result.__jqParentIndex = $this.attr('jq-repeat-parent-index');
}
$this.find('[jq-repeat]').each((idx, el)=>{
let templateIdx = result.nestedTemplates.length;
let template = `${el.outerHTML}`;
result.nestedTemplates.push(template);
$(el).replaceWith(`{{{ nestedTemplates.${templateIdx} }}}`);
});
result.__jqTemplate = $this[0].outerHTML;
$this.replaceWith( '<script type="x-tmpl-mustache" id="jq-repeat-holder-' + result.__jqRepeatId + '"><\/script>' );
result.$this = $('#jq-repeat-holder-' + result.__jqRepeatId);
Mustache.parse(result.__jqTemplate); // optional, speeds up future uses
for(let key in result){
Object.defineProperty(result, key, {
value: result[key],
writable: true,
enumerable: false,
configurable: true
});
}
var temp = $.scope[result.__jqRepeatId] || [];
$.scope[result.__jqRepeatId] = result;
for(let prop of Object.keys(temp)){
if(prop,Number.isInteger(Number(prop))){
$.scope[result.__jqRepeatId].push(temp[prop]);
}else{
$.scope[result.__jqRepeatId][prop] = temp[prop];
}
}
};
$( document ).ready( function(){
// Create an instance of MutationObserver and pass the callback function
const observer = new MutationObserver(function(mutationsList) {
mutationsList.forEach(mutation => {
if (mutation.type === 'childList') {
const addedNodes = mutation.addedNodes;
addedNodes.forEach(node => {
if (node.nodeType === Node.ELEMENT_NODE) {
const $el = $(node);
if ($el.is('[jq-repeat]')) {
make(node);
} else {
const toMake = [];
$el.find('[jq-repeat]').each((key, el) => {
if ($(el).parent().closest('[jq-repeat]').length) return;
toMake.push(el);
});
toMake.forEach(el => {
make(el);
});
}
}
});
}
});
});
// Start observing the document
observer.observe(document.body, { childList: true, subtree: true });
let toMake = [];
$( '[jq-repeat]' ).each(function(key, value){
if($(value).parent().closest('[jq-repeat]').length) return;
toMake.push(value);
});
for(let el of toMake){
make(el);
}
} );
})(jQuery, Mustache);
+1 -1
View File
@@ -1,7 +1,7 @@
'use strict';
const router = require('express').Router();
const { Auth } = require('../controller/auth');
const { Auth } = require('../models/auth');
router.post('/login', async function(req, res, next){
+11
View File
@@ -41,6 +41,17 @@ router.get('/lookup/:item', async function(req, res, next){
}
});
router.get('/lookupobj', async function(req, res, next){
try{
return res.json({
results: Model.lookUpObj,
});
}catch(error){
return next(error);
}
});
router.get('/:item', async function(req, res, next){
try{
+1 -1
View File
@@ -12,7 +12,7 @@ const values ={
// List of front end node modules to be served
const frontEndModules = ['bootstrap', 'mustache', 'jquery', '@fortawesome',
'moment', '@popper',
'moment', '@popper', 'jq-repeat',
];
// Server front end modules
+62
View File
@@ -0,0 +1,62 @@
'use strict';
const {Host} = require('../models/host');
const {SocketServerJson} = require('../utils/unix_socket_json');
const conf = require('../conf');
/**
* Host Lookup Service
*
* Unix socket server that handles host/domain lookup requests from OpenResty.
* This provides the bridge between nginx (Lua) and the Node.js host management system.
*
* Flow:
* 1. OpenResty sends domain lookup request via Unix socket
* 2. Service queries Host model (supports wildcards via lookup tree)
* 3. Returns host configuration (IP, port, SSL settings, etc.)
* 4. All values converted to strings for Redis compatibility
*
* Redis Compatibility:
* All object values are converted to strings before sending because:
* - Redis stores everything as strings
* - OpenResty's primary lookup path uses Redis directly (hgetall)
* - This socket is a fallback when Redis cache misses
* - Both paths must return identical data structures to Lua consumer
*/
const socket = new SocketServerJson({
socketFile: conf.socketFile,
onData: function(data, clientSocket) {
try{
// Try to match the requested host name using the lookup tree
let parentHost = Host.lookUp(data['domain']);
// If we don't have a match, return empty object
if(!parentHost) return clientSocket.write(JSON.stringify({}));
// If the matched host belongs to a wildcard domain, set wildcard_parent
// This allows child domains to use the parent's wildcard SSL certificate
if(!parentHost.wildcard_parent){
parentHost.wildcard_parent = parentHost.host;
Host.addCache(data['domain'], parentHost);
}
// Convert all values to strings for Redis compatibility
// OpenResty expects the same data format from both Redis and this socket
for(const [key, value] of Object.entries(parentHost)) {
parentHost[key] = String(value);
}
clientSocket.write(JSON.stringify(parentHost));
}catch(error){
console.error('services/host_lookup onData error', error);
}
},
onListen: function(){
console.log('Host lookup service listening on', conf.socketFile);
}
});
module.exports = {socket};
+32
View File
@@ -0,0 +1,32 @@
'use strict';
const {Host} = require('../models/host');
/**
* Host Scheduler Service
*
* Manages scheduled tasks for host-related operations:
* - Wildcard SSL certificate renewal checks
*
* Schedule:
* - Initial check: 30 seconds after application starts
* - Recurring checks: Every 24 hours (86400000ms)
*
* The checkWildcardForRenew method:
* - Iterates through all hosts in the system
* - Checks if wildcard certificates are expiring within 30 days
* - Automatically renews certificates that are approaching expiration
*/
// Initial wildcard cert check 30 seconds after app starts
// Delay allows the system to fully initialize before checking certs
setTimeout(Host.checkWildcardForRenew, 30000);
// Check wildcard certs once every 24 hours
// Ensures certificates are renewed well before expiration
setInterval(Host.checkWildcardForRenew, 86400000);
console.log('Host scheduler service initialized');
console.log('- Wildcard cert check: 30s after start, then every 24h');
module.exports = {};
+156
View File
@@ -0,0 +1,156 @@
# Test Suite
This project uses Node.js built-in test runner (requires Node 18+). No external testing dependencies required.
## Running Tests
```bash
# Run all tests
npm test
# Run only unit tests
npm run test:unit
# Run only integration tests
npm run test:integration
# Run tests in watch mode (auto-rerun on file changes)
npm run test:watch
```
## Test Structure
```
test/
├── unit/ # Unit tests for isolated components
│ ├── callback_queue.test.js
│ ├── host_lookup.test.js
│ └── unix_socket.test.js
├── integration/ # Integration tests for complex interactions
│ └── dns_provider.test.js
└── helpers/ # Test utilities and contracts
└── dns_provider_contract.js
```
## What We Test
### Unit Tests
**callback_queue.test.js**
- Callback registration and invocation
- Multiple callbacks with arguments
- Error handling
**host_lookup.test.js**
- Host lookup tree algorithm
- Wildcard matching (single and double)
- Exact match priority
- Edge cases (no match, empty input, etc.)
**unix_socket.test.js**
- Unix socket server creation
- JSON message parsing
- Partial data buffering
- Multiple connections
- Error handling
### Integration Tests
**dns_provider.test.js**
- DNS provider contract compliance
- All existing providers (CloudFlare, DigitalOcean, PorkBun)
- Method signatures
- Key mapping
- Type validation
## Adding a New DNS Provider
When you add a new DNS provider, you MUST add tests to ensure it meets the contract:
1. Create your provider class extending `DnsApi` in `models/dns_provider/yourprovider.js`
2. Add a test block in `test/integration/dns_provider.test.js`:
```javascript
describe('YourProvider Provider', () => {
const YourProvider = require('../../models/dns_provider/yourprovider');
test('should meet DNS provider contract', () => {
const mockCredentials = {api_key: 'mock-key'};
const instance = validateDnsProviderContract(YourProvider, mockCredentials);
assert.ok(instance, 'YourProvider should be instantiated');
});
test('should have correct _keyMap structure', () => {
// Test your specific credential requirements
assert.ok(YourProvider._keyMap.api_key);
assert.strictEqual(YourProvider._keyMap.api_key.type, 'string');
assert.strictEqual(YourProvider._keyMap.api_key.isRequired, true);
});
test('should have valid method signatures', () => {
const instance = new YourProvider({api_key: 'mock'});
validateMethodSignatures(instance);
});
test('should validate key mapping', () => {
const instance = new YourProvider({api_key: 'mock'});
validateKeyMapping(instance);
});
test('should validate type checking', () => {
const instance = new YourProvider({api_key: 'mock'});
validateTypeChecking(instance);
});
});
```
3. Run tests to verify compliance:
```bash
npm run test:integration
```
## DNS Provider Contract
All DNS providers must:
1. Extend `DnsApi` base class
2. Define static `_keyMap` with required credentials
3. Define static display properties: `displayName`, `displayIconHtml`, `displayIconUni`
4. Implement required methods:
- `listDomains()` - Returns array of `{domain, zoneId}`
- `getRecords(domain, options)` - Returns array of DNS records
- `createRecord(domain, options)` - Creates a record
- `deleteRecords(domain, options)` - Deletes matching records
5. Define `__apiKeyMap` to translate between class keys and API keys
6. Implement or inherit `__typeCheck()` for record type validation
7. Throw appropriate errors from `this.errors` object
## CI/CD Integration
Tests can be run in GitHub Actions, GitLab CI, or any CI/CD system:
```yaml
# Example GitHub Actions workflow
- name: Run tests
run: npm test
```
## Philosophy
We test **custom logic**, not third-party code:
- YES: Test our host lookup algorithm
- YES: Test our socket buffering logic
- YES: Test DNS provider contracts
- NO: Don't test Express.js routing
- NO: Don't test the Redis ORM
- NO: Don't test external DNS APIs (use mocks)
## Notes
- Tests use Node's built-in `node:test` and `node:assert` modules
- No external testing framework needed
- Tests are fast and run in parallel by default
- Mock external services (Redis, DNS APIs) to avoid network calls
- Focus on testing business logic, not infrastructure
@@ -0,0 +1,285 @@
'use strict';
const assert = require('node:assert');
const {DnsApi} = require('../../models/dns_provider/common');
/**
* DNS Provider Contract Test Helper
*
* This module provides a contract test suite that validates a DNS provider
* implementation meets all requirements. Use this when adding new DNS providers
* to ensure they implement the required interface correctly.
*
* Required static properties:
* - _keyMap: Object defining required API credentials/config
* - displayName: String for UI display
* - displayIconHtml: SVG markup for provider icon
* - displayIconUni: Unicode icon fallback
*
* Required instance methods:
* - listDomains(): Returns array of {domain, zoneId}
* - getRecords(domain, options): Returns array of DNS records
* - createRecord(domain, options): Creates a record, returns created record
* - deleteRecords(domain, options): Deletes matching records
*
* Required behavior:
* - Must extend DnsApi base class
* - Must throw errors.unauthorized() on auth failures
* - Must implement __apiKeyMap for key translation
* - Must validate record types via __typeCheck
*/
/**
* Validates a DNS provider class meets the contract
*
* @param {Class} ProviderClass - The DNS provider class to validate
* @param {Object} mockCredentials - Mock credentials for testing
* @returns {void}
* @throws {AssertionError} If provider doesn't meet contract
*/
function validateDnsProviderContract(ProviderClass, mockCredentials) {
// Test 1: Must extend DnsApi
assert.ok(
ProviderClass.prototype instanceof DnsApi,
`${ProviderClass.name} must extend DnsApi base class`
);
// Test 2: Must have static _keyMap
assert.ok(
ProviderClass._keyMap,
`${ProviderClass.name} must define static _keyMap`
);
assert.strictEqual(
typeof ProviderClass._keyMap,
'object',
`${ProviderClass.name}._keyMap must be an object`
);
// Test 3: Must have display properties
assert.ok(
ProviderClass.displayName,
`${ProviderClass.name} must define static displayName`
);
assert.ok(
ProviderClass.displayIconHtml,
`${ProviderClass.name} must define static displayIconHtml`
);
assert.ok(
ProviderClass.displayIconUni,
`${ProviderClass.name} must define static displayIconUni`
);
// Test 4: Can be instantiated
let instance;
assert.doesNotThrow(
() => {
instance = new ProviderClass(mockCredentials);
},
`${ProviderClass.name} must be instantiable with mock credentials`
);
// Test 5: Must have required methods
const requiredMethods = [
'listDomains',
'getRecords',
'createRecord',
'deleteRecords'
];
for(let method of requiredMethods) {
assert.strictEqual(
typeof instance[method],
'function',
`${ProviderClass.name} must implement ${method}() method`
);
}
// Test 6: Must have __apiKeyMap for key translation
assert.ok(
instance.hasOwnProperty('__apiKeyMap') || instance.constructor.prototype.hasOwnProperty('__apiKeyMap'),
`${ProviderClass.name} must define __apiKeyMap property`
);
// Test 7: Must have __typeCheck method (inherited or overridden)
assert.strictEqual(
typeof instance.__typeCheck,
'function',
`${ProviderClass.name} must have __typeCheck method`
);
// Test 8: Must have error methods (inherited from DnsApi)
assert.ok(
instance.errors,
`${ProviderClass.name} must have errors object`
);
assert.strictEqual(
typeof instance.errors.unauthorized,
'function',
`${ProviderClass.name} must have errors.unauthorized method`
);
assert.strictEqual(
typeof instance.errors.invalidInput,
'function',
`${ProviderClass.name} must have errors.invalidInput method`
);
assert.strictEqual(
typeof instance.errors.other,
'function',
`${ProviderClass.name} must have errors.other method`
);
// Test 9: info() method should return expected structure
const info = ProviderClass.info();
assert.ok(info.displayName, 'info() must include displayName');
assert.ok(info.displayIconHtml, 'info() must include displayIconHtml');
assert.ok(info.displayIconUni, 'info() must include displayIconUni');
assert.ok(info.fields, 'info() must include fields');
// Test 10: toJSON() should work
const json = instance.toJSON();
assert.ok(json.displayName, 'toJSON() must include displayName');
return instance;
}
/**
* Validates method signatures for a DNS provider instance
*
* @param {Object} instance - Instance of DNS provider
* @param {Object} mockDomain - Mock domain object with {domain, zoneId}
* @returns {void}
*/
function validateMethodSignatures(instance, mockDomain = {domain: 'example.com', zoneId: 'mock-zone'}) {
const className = instance.constructor.name;
// These tests just verify the methods accept the expected parameters
// and return promises (actual API calls would require real credentials)
// We catch and suppress errors since these calls will fail with mock credentials
// listDomains() should return a promise
const listDomainsResult = instance.listDomains();
assert.ok(
listDomainsResult instanceof Promise,
`${className}.listDomains() must return a Promise`
);
listDomainsResult.catch(() => {}); // Suppress unhandled rejection
// getRecords(domain, options) should return a promise
const getRecordsResult = instance.getRecords(mockDomain, {type: 'A'});
assert.ok(
getRecordsResult instanceof Promise,
`${className}.getRecords() must return a Promise`
);
getRecordsResult.catch(() => {}); // Suppress unhandled rejection
// createRecord(domain, options) should return a promise
const createRecordResult = instance.createRecord(mockDomain, {
type: 'TXT',
name: 'test',
data: 'test-value'
});
assert.ok(
createRecordResult instanceof Promise,
`${className}.createRecord() must return a Promise`
);
createRecordResult.catch(() => {}); // Suppress unhandled rejection
// deleteRecords(domain, options) should return a promise
const deleteRecordsResult = instance.deleteRecords(mockDomain, {
type: 'TXT',
name: 'test'
});
assert.ok(
deleteRecordsResult instanceof Promise,
`${className}.deleteRecords() must return a Promise`
);
deleteRecordsResult.catch(() => {}); // Suppress unhandled rejection
}
/**
* Validates __parseOptions and __parseRes behavior
*
* @param {Object} instance - Instance of DNS provider
* @returns {void}
*/
function validateKeyMapping(instance) {
const className = instance.constructor.name;
// Test __parseOptions normalizes keys
if(Object.keys(instance.__apiKeyMap).length > 0) {
const testOptions = {type: 'A'};
// Add a class key that should be mapped to API key
const [apiKey, clsKey] = Object.entries(instance.__apiKeyMap)[0];
testOptions[clsKey] = 'test-value';
const parsed = instance.__parseOptions(testOptions);
assert.ok(
parsed.hasOwnProperty(apiKey),
`${className}.__parseOptions() should map '${clsKey}' to '${apiKey}'`
);
assert.strictEqual(
parsed[clsKey],
undefined,
`${className}.__parseOptions() should remove class key '${clsKey}' after mapping`
);
}
// Test __parseRes normalizes response keys
const testResponse = [];
const [apiKey, clsKey] = Object.entries(instance.__apiKeyMap)[0] || ['content', 'data'];
testResponse.push({[apiKey]: 'test-value', name: 'test.example.com'});
const parsedRes = instance.__parseRes(testResponse);
if(Object.keys(instance.__apiKeyMap).length > 0) {
assert.ok(
parsedRes[0].hasOwnProperty(clsKey),
`${className}.__parseRes() should map '${apiKey}' to '${clsKey}'`
);
}
}
/**
* Validates type checking behavior
*
* @param {Object} instance - Instance of DNS provider
* @returns {void}
*/
function validateTypeChecking(instance) {
const className = instance.constructor.name;
const validTypes = ['A', 'MX', 'CNAME', 'ALIAS', 'TXT', 'NS', 'AAAA', 'SRV', 'TLSA', 'CAA'];
// Valid types should not throw
for(let type of validTypes) {
assert.doesNotThrow(
() => instance.__typeCheck(type),
`${className}.__typeCheck() should accept valid type '${type}'`
);
}
// Invalid type should throw
assert.throws(
() => instance.__typeCheck('INVALID'),
/Invalid.*type/i,
`${className}.__typeCheck() should reject invalid types`
);
}
module.exports = {
validateDnsProviderContract,
validateMethodSignatures,
validateKeyMapping,
validateTypeChecking
};
@@ -0,0 +1,190 @@
'use strict';
const {describe, test} = require('node:test');
const assert = require('node:assert');
const {
validateDnsProviderContract,
validateMethodSignatures,
validateKeyMapping,
validateTypeChecking
} = require('../helpers/dns_provider_contract');
/**
* DNS Provider Integration Tests
*
* These tests validate that each DNS provider implementation meets
* the required contract. When adding a new DNS provider:
*
* 1. Add a new describe block for your provider
* 2. Import your provider class
* 3. Run the contract validation tests
* 4. Add any provider-specific tests as needed
*
* The contract tests will verify:
* - Class extends DnsApi
* - Required static properties are defined
* - Required methods are implemented
* - Error handling is correct
* - Key mapping works correctly
* - Type validation is implemented
*/
describe('DNS Provider Contract Compliance', () => {
describe('CloudFlare Provider', () => {
const CloudFlare = require('../../models/dns_provider/cloudflare');
test('should meet DNS provider contract', () => {
const mockCredentials = {token: 'mock-token-for-testing'};
const instance = validateDnsProviderContract(CloudFlare, mockCredentials);
assert.ok(instance, 'CloudFlare provider should be instantiated');
});
test('should have correct _keyMap structure', () => {
assert.ok(CloudFlare._keyMap.token, 'Should require token');
assert.strictEqual(CloudFlare._keyMap.token.type, 'string');
assert.strictEqual(CloudFlare._keyMap.token.isRequired, true);
assert.strictEqual(CloudFlare._keyMap.token.isPrivate, true);
});
test('should have correct display properties', () => {
assert.strictEqual(CloudFlare.displayName, 'CloudFlare');
assert.ok(CloudFlare.displayIconHtml.includes('svg'));
assert.ok(CloudFlare.displayIconUni);
});
test('should map content to data', () => {
const instance = new CloudFlare({token: 'mock'});
assert.deepStrictEqual(instance.__apiKeyMap, {'content': 'data'});
});
test('should have valid method signatures', () => {
const instance = new CloudFlare({token: 'mock'});
validateMethodSignatures(instance);
});
test('should validate key mapping', () => {
const instance = new CloudFlare({token: 'mock'});
validateKeyMapping(instance);
});
test('should validate type checking', () => {
const instance = new CloudFlare({token: 'mock'});
validateTypeChecking(instance);
});
});
describe('DigitalOcean Provider', () => {
const DigitalOcean = require('../../models/dns_provider/digitalocean');
test('should meet DNS provider contract', () => {
const mockCredentials = {token: 'mock-token-for-testing'};
const instance = validateDnsProviderContract(DigitalOcean, mockCredentials);
assert.ok(instance, 'DigitalOcean provider should be instantiated');
});
test('should have correct _keyMap structure', () => {
assert.ok(DigitalOcean._keyMap.token, 'Should require token');
assert.strictEqual(DigitalOcean._keyMap.token.type, 'string');
assert.strictEqual(DigitalOcean._keyMap.token.isRequired, true);
});
test('should have valid method signatures', () => {
const instance = new DigitalOcean({token: 'mock'});
validateMethodSignatures(instance);
});
test('should validate key mapping', () => {
const instance = new DigitalOcean({token: 'mock'});
validateKeyMapping(instance);
});
test('should validate type checking', () => {
const instance = new DigitalOcean({token: 'mock'});
validateTypeChecking(instance);
});
});
describe('PorkBun Provider', () => {
const PorkBun = require('../../models/dns_provider/porkbun');
test('should meet DNS provider contract', () => {
const mockCredentials = {
apiKey: 'mock-api-key',
secretApiKey: 'mock-secret-key'
};
const instance = validateDnsProviderContract(PorkBun, mockCredentials);
assert.ok(instance, 'PorkBun provider should be instantiated');
});
test('should have correct _keyMap structure', () => {
assert.ok(PorkBun._keyMap.apiKey, 'Should require apiKey');
assert.ok(PorkBun._keyMap.secretApiKey, 'Should require secretApiKey');
assert.strictEqual(PorkBun._keyMap.apiKey.type, 'string');
assert.strictEqual(PorkBun._keyMap.apiKey.isRequired, true);
assert.strictEqual(PorkBun._keyMap.secretApiKey.type, 'string');
assert.strictEqual(PorkBun._keyMap.secretApiKey.isRequired, true);
});
test('should have valid method signatures', () => {
const instance = new PorkBun({
apiKey: 'mock-api-key',
secretApiKey: 'mock-secret-key'
});
validateMethodSignatures(instance);
});
test('should validate key mapping', () => {
const instance = new PorkBun({
apiKey: 'mock-api-key',
secretApiKey: 'mock-secret-key'
});
validateKeyMapping(instance);
});
test('should validate type checking', () => {
const instance = new PorkBun({
apiKey: 'mock-api-key',
secretApiKey: 'mock-secret-key'
});
validateTypeChecking(instance);
});
});
});
/**
* Example: How to add tests for a new DNS provider
*
* describe('NewProvider Provider', () => {
* const NewProvider = require('../../models/dns_provider/newprovider');
*
* test('should meet DNS provider contract', () => {
* const mockCredentials = {api_key: 'mock-key'};
* const instance = validateDnsProviderContract(NewProvider, mockCredentials);
* assert.ok(instance, 'NewProvider should be instantiated');
* });
*
* test('should have correct _keyMap structure', () => {
* // Verify your provider's specific credential requirements
* assert.ok(NewProvider._keyMap.api_key);
* });
*
* test('should have valid method signatures', () => {
* const instance = new NewProvider({api_key: 'mock'});
* validateMethodSignatures(instance);
* });
*
* test('should validate key mapping', () => {
* const instance = new NewProvider({api_key: 'mock'});
* validateKeyMapping(instance);
* });
*
* test('should validate type checking', () => {
* const instance = new NewProvider({api_key: 'mock'});
* validateTypeChecking(instance);
* });
* });
*/
+138
View File
@@ -0,0 +1,138 @@
'use strict';
const {describe, test} = require('node:test');
const assert = require('node:assert');
const {CallbackQueue} = require('../../utils/callback_queue');
/**
* Tests for CallbackQueue utility
*
* CallbackQueue manages multiple callbacks for a single event, allowing
* multiple listeners to be registered and called with the same arguments.
*/
describe('CallbackQueue', () => {
test('should initialize with a single callback function', () => {
const callback = () => {};
const queue = new CallbackQueue(callback);
assert.strictEqual(queue.__callbacks.length, 1);
assert.strictEqual(queue.__callbacks[0], callback);
});
test('should initialize with an array of callbacks', () => {
const callback1 = () => {};
const callback2 = () => {};
const queue = new CallbackQueue([callback1, callback2]);
assert.strictEqual(queue.__callbacks.length, 2);
assert.strictEqual(queue.__callbacks[0], callback1);
assert.strictEqual(queue.__callbacks[1], callback2);
});
test('should initialize with empty queue when no callback provided', () => {
const queue = new CallbackQueue();
assert.strictEqual(queue.__callbacks.length, 0);
});
test('should push a function to the queue', () => {
const queue = new CallbackQueue();
const callback = () => {};
queue.push(callback);
assert.strictEqual(queue.__callbacks.length, 1);
assert.strictEqual(queue.__callbacks[0], callback);
});
test('should ignore non-function values when pushing', () => {
const queue = new CallbackQueue();
queue.push('not a function');
queue.push(123);
queue.push(null);
queue.push(undefined);
queue.push({});
assert.strictEqual(queue.__callbacks.length, 0);
});
test('should call all callbacks with provided arguments', () => {
const results = [];
const callback1 = (a, b) => results.push(['cb1', a, b]);
const callback2 = (a, b) => results.push(['cb2', a, b]);
const queue = new CallbackQueue([callback1, callback2]);
queue.call('arg1', 'arg2');
assert.strictEqual(results.length, 2);
assert.deepStrictEqual(results[0], ['cb1', 'arg1', 'arg2']);
assert.deepStrictEqual(results[1], ['cb2', 'arg1', 'arg2']);
});
test('should call callbacks with no arguments', () => {
let called = false;
const callback = () => { called = true; };
const queue = new CallbackQueue(callback);
queue.call();
assert.strictEqual(called, true);
});
test('should handle callbacks that throw errors without stopping other callbacks', () => {
const results = [];
const callback1 = () => results.push('cb1');
const callback2 = () => { throw new Error('Test error'); };
const callback3 = () => results.push('cb3');
const queue = new CallbackQueue([callback1, callback2, callback3]);
// The error will be thrown but shouldn't stop execution
assert.throws(() => {
queue.call();
}, /Test error/);
// Only cb1 should have been called before the error
assert.strictEqual(results.length, 1);
assert.strictEqual(results[0], 'cb1');
});
test('should call callbacks without specific context', () => {
let receivedThis = null;
const callback = function() { receivedThis = this; };
const queue = new CallbackQueue(callback);
queue.call();
// Callbacks are called without binding, so 'this' is undefined in strict mode
assert.strictEqual(receivedThis, undefined);
});
test('should allow adding callbacks after initialization', () => {
const results = [];
const callback1 = () => results.push('cb1');
const callback2 = () => results.push('cb2');
const queue = new CallbackQueue(callback1);
queue.push(callback2);
queue.call();
assert.strictEqual(results.length, 2);
assert.deepStrictEqual(results, ['cb1', 'cb2']);
});
test('should work with callbacks that return values', () => {
const callback1 = () => 'result1';
const callback2 = () => 'result2';
const queue = new CallbackQueue([callback1, callback2]);
// Note: call() doesn't return values, it just executes callbacks
// This test verifies callbacks can return values without breaking
assert.doesNotThrow(() => {
queue.call();
});
});
});
+213
View File
@@ -0,0 +1,213 @@
'use strict';
const {describe, test, before} = require('node:test');
const assert = require('node:assert');
/**
* Tests for Host lookup algorithm
*
* The Host.lookUp method implements a complex tree-based lookup system
* that supports exact matches, single wildcards (*), and double wildcards (**).
*
* Pattern matching priority (highest to lowest):
* 1. Exact match (example.com)
* 2. Single wildcard (*.example.com matches any.example.com)
* 3. Double wildcard (**.example.com matches any.sub.domain.example.com)
*
* These tests validate the lookup algorithm without requiring a Redis connection.
*/
describe('Host Lookup Algorithm', () => {
let Host;
before(async () => {
// Mock the Host class and build a test lookup tree
Host = createMockHostClass();
await populateTestData(Host);
});
test('should match exact host', () => {
const result = Host.lookUp('payments.718it.biz');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'payments.718it.biz');
});
test('should return undefined for non-existent host', () => {
const result = Host.lookUp('sd.blah.test.vm42.com');
assert.strictEqual(result, undefined);
});
test('should match double wildcard at any depth', () => {
const result = Host.lookUp('payments.test.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'payments.**');
});
test('should match double wildcard with multiple subdomains', () => {
const result = Host.lookUp('test.sample.other.exmaple.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, '**.exmaple.com');
});
test('should prefer exact match over wildcard', () => {
const result = Host.lookUp('stan.test.vm42.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'stan.test.vm42.com');
});
test('should match at root level', () => {
const result = Host.lookUp('test.vm42.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'test.vm42.com');
});
test('should match single wildcard', () => {
const result = Host.lookUp('blah.test.vm42.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, '*.test.vm42.com');
});
test('should match double wildcard for top-level domain queries', () => {
const result = Host.lookUp('payments.example.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'payments.**');
});
test('should match single wildcard in middle of domain', () => {
const result = Host.lookUp('info.wma.users.718it.biz');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'info.*.users.718it.biz');
});
test('should return undefined when single wildcard does not match', () => {
const result = Host.lookUp('infof.users.718it.biz');
assert.strictEqual(result, undefined);
});
test('should return undefined for non-existent TLD', () => {
const result = Host.lookUp('blah.biz');
assert.strictEqual(result, undefined);
});
test('should match multiple single wildcards', () => {
const result = Host.lookUp('test.1.2.718it.net');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'test.*.*.718it.net');
});
test('should match exact subdomain', () => {
const result = Host.lookUp('test1.exmaple.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'test1.exmaple.com');
});
test('should match single wildcard when exact not found', () => {
const result = Host.lookUp('other.exmaple.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, '*.exmaple.com');
});
test('should match double wildcard with subdomain prefix', () => {
const result = Host.lookUp('info.payments.example.com');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, 'info.**');
});
test('should match bare domain', () => {
const result = Host.lookUp('718it.biz');
assert.ok(result, 'Should find a match');
assert.strictEqual(result.host, '718it.biz');
});
test('should handle single-part domain', () => {
const result = Host.lookUp('localhost');
assert.strictEqual(result, undefined);
});
test('should handle empty string', () => {
const result = Host.lookUp('');
assert.strictEqual(result, undefined);
});
test('should be case-sensitive', () => {
const result = Host.lookUp('PAYMENTS.718it.biz');
assert.strictEqual(result, undefined);
});
});
/**
* Creates a mock Host class with just the lookUp functionality
* This allows us to test the algorithm without Redis dependencies
*/
function createMockHostClass() {
return class MockHost {
static lookUpObj = {};
static __lookUpIsReady = false;
static lookUp(host) {
// This is the exact implementation from models/host.js lines 324-357
let place = this.lookUpObj;
let last_resort = {};
for(let fragment of host.split('.').reverse()){
if(place['**']) last_resort = place['**'];
if({...last_resort, ...place}[fragment]){
place = {...last_resort, ...place}[fragment];
}else if(place['*']){
place = place['*']
}else if(last_resort){
place = last_resort;
}
}
if(place && place['#record']) return place['#record'];
}
};
}
/**
* Populates the mock Host class with test data
* Builds the lookup tree structure based on test cases from models/host.js
*/
async function populateTestData(Host) {
// Test data based on the commented test cases in models/host.js
const testHosts = [
'payments.718it.biz',
'payments.**',
'**.exmaple.com',
'stan.test.vm42.com',
'test.vm42.com',
'*.test.vm42.com',
'info.*.users.718it.biz',
'test.*.*.718it.net',
'test1.exmaple.com',
'*.exmaple.com',
'info.**',
'718it.biz',
];
Host.lookUpObj = {};
for(let host of testHosts){
let fragments = host.split('.');
let pointer = Host.lookUpObj;
while(fragments.length){
let fragment = fragments.pop();
if(!pointer[fragment]){
pointer[fragment] = {};
}
if(fragments.length === 0){
pointer[fragment]['#record'] = {host};
}
pointer = pointer[fragment];
}
}
Host.__lookUpIsReady = true;
}
+306
View File
@@ -0,0 +1,306 @@
'use strict';
const {describe, test, after} = require('node:test');
const assert = require('node:assert');
const net = require('net');
const fs = require('fs');
const path = require('path');
const {SocketServerJson} = require('../../utils/unix_socket_json');
/**
* Tests for Unix Socket JSON Server
*
* Tests the socket server's ability to:
* - Accept connections on Unix socket
* - Parse complete JSON messages
* - Handle partial JSON data (buffering)
* - Trigger callbacks correctly
* - Clean up socket files
*/
describe('Unix Socket JSON Server', () => {
// Use a test-specific socket file
const testSocketFile = path.join('/tmp', `test-socket-${Date.now()}.sock`);
let activeServers = [];
after(() => {
// Cleanup: close all servers and remove socket files
activeServers.forEach(server => {
try {
if(server.socket) server.socket.close();
} catch(e) {}
});
try {
if(fs.existsSync(testSocketFile)) {
fs.unlinkSync(testSocketFile);
}
} catch(e) {}
});
test('should create and listen on Unix socket', (t, done) => {
const server = new SocketServerJson({
socketFile: testSocketFile,
onListen: () => {
assert.ok(fs.existsSync(testSocketFile), 'Socket file should exist');
server.socket.close();
done();
}
});
activeServers.push(server);
});
test('should parse complete JSON message', (t, done) => {
const testData = {message: 'hello', value: 123};
let receivedData = null;
const server = new SocketServerJson({
socketFile: testSocketFile + '-json',
onData: (data, clientSocket) => {
receivedData = data;
clientSocket.end();
},
onListen: () => {
// Connect and send JSON
const client = net.createConnection(testSocketFile + '-json', () => {
client.write(JSON.stringify(testData));
});
client.on('close', () => {
assert.deepStrictEqual(receivedData, testData);
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-json')) {
fs.unlinkSync(testSocketFile + '-json');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
test('should handle partial JSON data', (t, done) => {
const testData = {message: 'hello world', value: 456, nested: {foo: 'bar'}};
const jsonString = JSON.stringify(testData);
let receivedData = null;
const server = new SocketServerJson({
socketFile: testSocketFile + '-partial',
onData: (data, clientSocket) => {
receivedData = data;
clientSocket.end();
},
onListen: () => {
const client = net.createConnection(testSocketFile + '-partial', () => {
// Send JSON in chunks to simulate partial data
const chunk1 = jsonString.slice(0, 10);
const chunk2 = jsonString.slice(10);
client.write(chunk1);
// Wait a bit then send the rest
setTimeout(() => {
client.write(chunk2);
}, 10);
});
client.on('close', () => {
assert.deepStrictEqual(receivedData, testData);
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-partial')) {
fs.unlinkSync(testSocketFile + '-partial');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
test('should call multiple onData callbacks', (t, done) => {
const testData = {test: 'data'};
const callbackResults = [];
const server = new SocketServerJson({
socketFile: testSocketFile + '-multi',
onData: [
(data) => callbackResults.push('callback1'),
(data) => callbackResults.push('callback2'),
],
onListen: () => {
const client = net.createConnection(testSocketFile + '-multi', () => {
client.write(JSON.stringify(testData));
client.end();
});
client.on('close', () => {
assert.strictEqual(callbackResults.length, 2);
assert.strictEqual(callbackResults[0], 'callback1');
assert.strictEqual(callbackResults[1], 'callback2');
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-multi')) {
fs.unlinkSync(testSocketFile + '-multi');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
test('should provide client socket to callbacks', (t, done) => {
const testData = {request: 'test'};
const responseData = {response: 'success'};
let receivedResponse = '';
const server = new SocketServerJson({
socketFile: testSocketFile + '-response',
onData: (data, clientSocket) => {
// Echo back a response
clientSocket.write(JSON.stringify(responseData));
clientSocket.end();
},
onListen: () => {
const client = net.createConnection(testSocketFile + '-response', () => {
client.write(JSON.stringify(testData));
});
client.on('data', (data) => {
receivedResponse += data.toString();
});
client.on('close', () => {
assert.deepStrictEqual(JSON.parse(receivedResponse), responseData);
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-response')) {
fs.unlinkSync(testSocketFile + '-response');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
test('should clean up existing socket file on startup', (t, done) => {
const socketPath = testSocketFile + '-cleanup';
// Create a stale socket file
fs.writeFileSync(socketPath, 'stale');
const server = new SocketServerJson({
socketFile: socketPath,
onListen: () => {
// Should have removed the old file and created a new socket
assert.ok(fs.existsSync(socketPath));
const stats = fs.statSync(socketPath);
assert.ok(stats.isSocket(), 'Should be a socket, not a regular file');
server.socket.close();
try {
if(fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
} catch(e) {}
done();
}
});
activeServers.push(server);
});
test('should handle multiple sequential messages', (t, done) => {
const messages = [
{id: 1, text: 'first'},
{id: 2, text: 'second'},
{id: 3, text: 'third'}
];
const receivedMessages = [];
const server = new SocketServerJson({
socketFile: testSocketFile + '-sequential',
onData: (data, clientSocket) => {
receivedMessages.push(data);
if(receivedMessages.length === messages.length) {
clientSocket.end();
}
},
onListen: () => {
const client = net.createConnection(testSocketFile + '-sequential', () => {
// Send messages with delays to simulate separate events
messages.forEach((msg, index) => {
setTimeout(() => {
client.write(JSON.stringify(msg));
}, index * 10);
});
});
client.on('close', () => {
assert.strictEqual(receivedMessages.length, messages.length);
assert.deepStrictEqual(receivedMessages[0], messages[0]);
assert.deepStrictEqual(receivedMessages[1], messages[1]);
assert.deepStrictEqual(receivedMessages[2], messages[2]);
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-sequential')) {
fs.unlinkSync(testSocketFile + '-sequential');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
test('should buffer data and parse when complete JSON received', (t, done) => {
// This test verifies the buffer accumulates until valid JSON is formed
// Note: Once malformed JSON enters buffer, it cannot recover
const testData = {test: 'buffering', value: 999};
const jsonString = JSON.stringify(testData);
let receivedData = null;
const server = new SocketServerJson({
socketFile: testSocketFile + '-buffer',
onData: (data, clientSocket) => {
receivedData = data;
clientSocket.end();
},
onListen: () => {
const client = net.createConnection(testSocketFile + '-buffer', () => {
// Send in three parts to test buffering
const part1 = jsonString.slice(0, 8);
const part2 = jsonString.slice(8, 16);
const part3 = jsonString.slice(16);
client.write(part1);
setTimeout(() => {
client.write(part2);
setTimeout(() => {
client.write(part3);
}, 5);
}, 5);
});
client.on('close', () => {
assert.deepStrictEqual(receivedData, testData);
server.socket.close();
try {
if(fs.existsSync(testSocketFile + '-buffer')) {
fs.unlinkSync(testSocketFile + '-buffer');
}
} catch(e) {}
done();
});
}
});
activeServers.push(server);
});
});
+2
View File
@@ -38,6 +38,7 @@ function ModelPs(model){
publish(propKey, res, ...args);
}).catch(function(error){
// console.error("Error PS", model.name, propKey, error)
console.log('toDo, publish errors...');
});
}else{
@@ -45,6 +46,7 @@ function ModelPs(model){
}
return res;
}catch(error){
// console.error("Error PS", model.name, propKey, error)
console.log("toDo, publish errors...");
}
}
+120
View File
@@ -0,0 +1,120 @@
'use strict';
const net = require('net');
const fs = require('fs');
const {CallbackQueue} = require('../utils/callback_queue');
/**
* SocketServerJson
*
* A generic Unix socket server that handles JSON message communication.
* Automatically manages socket file lifecycle (cleanup, permissions).
* Uses callback queues for event handling to support multiple listeners.
*
* Features:
* - Automatic socket file cleanup on startup
* - Configurable file permissions (default 777 for container use)
* - JSON message parsing with buffering for partial data
* - Event callbacks for data, errors, and connection lifecycle
*/
class SocketServerJson {
constructor(args){
this.socketFile = args.socketFile;
this.onData = new CallbackQueue(args.onData, this);
this.onListen = new CallbackQueue(args.onListen, this);
this.onError = new CallbackQueue(args.onError);
this.onClientNew = new CallbackQueue(args.onClientNew);
this.onClientClose = new CallbackQueue(args.onClientClose);
this.onClientError = new CallbackQueue(args.onClientError);
// Set socket file permissions after listening
// 777 is acceptable here for single-use container environments
// Wrapped in try-catch as chmod may fail in test/restricted environments
this.onListen.push(() => {
try {
fs.chmodSync(this.socketFile, '777');
} catch(err) {
// Chmod may fail in test environments or certain filesystems
// Socket will still work with default permissions
}
});
this.listen();
}
/**
* Removes existing socket file if present before creating new server.
* Prevents "address already in use" errors from stale socket files.
*/
__resetSocketFile(callback){
let instance = this;
fs.stat(this.socketFile, function (err, stats) {
if (stats) {
fs.unlink(instance.socketFile, function(err){
if(err){
// This should never happen
console.error(err);
}
callback(...arguments);
});
}else{
callback();
}
});
}
/**
* Sets up the Unix socket server and registers event handlers.
*
* Data handling:
* - Buffers incoming data to handle partial JSON messages
* - Attempts to parse buffer as JSON after each data event
* - Clears buffer only after successful parse
* - Silent failure on parse errors (waits for more data)
*/
__setUpServer(){
let instance = this;
this.socket = net.createServer();
this.socket.on('connection', function(clientSocket){
let buffer = '';
clientSocket.on('data', function(data){
buffer += data.toString();
try{
// Parse buffer (not just current data chunk) to handle partial messages
instance.onData.call(JSON.parse(buffer), clientSocket);
buffer = '';
}catch(error){
// Parse failed - likely incomplete JSON, wait for more data
// Buffer persists until complete JSON is received
}
});
clientSocket.on('close', instance.onClientClose.call.bind(instance.onClientClose));
clientSocket.on('error', instance.onClientError.call.bind(instance.onClientError));
});
this.socket.on('error', this.onError.call.bind(this.onError));
this.socket.on('listening', this.onListen.call.bind(this.onListen));
}
/**
* Starts the Unix socket server.
* Cleans up any existing socket file before listening.
*/
listen(){
let instance = this;
this.__setUpServer();
this.__resetSocketFile(function(){
instance.socket.listen(instance.socketFile);
});
}
}
module.exports = {SocketServerJson};
+3 -6
View File
@@ -179,14 +179,11 @@
<h3><img height="32px" src="{{ displayIconHtml }}"/> {{name}} </h3>
</div>
<div>
<b jq-repeat='Domain_{{id}}' jq-repeat-index="domain" class="me-2 my-1 badge text-bg-info rounded-pill fs-6">
{{#domains}}
<b class="me-2 my-1 badge text-bg-info rounded-pill fs-6">
{{domain}}
</b>
<script type="text/javascript">
try{
$.scope.Domain_{{id}}.push(...{{{domainsString}}});
}catch{}
</script>
{{/domains}}
</div>
</div>
<div class="card-footer">
+121 -25
View File
@@ -13,6 +13,7 @@
div.form-group{
margin-bottom: 1em;
}
/* my Div class for my search bar */
.search-wrapper {
display: flex;
@@ -20,6 +21,7 @@
align-items: center;
margin-top: 10px;
}
/* The input bar */
input {
font-size: 1rem;
@@ -28,17 +30,26 @@
border-top-right-radius: 5px !important;
border-bottom-right-radius: 5px !important;
}
.challengeType-container {
pointer-events: none; /* Prevents clicking */
opacity: 0.5; /* Greys it out */
filter: grayscale(1); /* Removes blue/color tint */
cursor: not-allowed;
}
</style>
<script type="text/javascript">
var $editHostForm;
// Parse the JSON object for a host to something the UI wants
function hostParseRow(host) {
host['updated_on_text'] = moment(host['updated_on'], "x").fromNow();
host['wildcard_expires_text'] = moment(host['wildcard_expires'], "x").fromNow();
host['targetssl_text'] = host['targetssl'] ? 'https://' : 'http://';
host['forcessl_text'] = host['forcessl'] ? 'https://' : 'http://';
host['wildcard_text'] = host['is_wildcard'] ? host['wildcard_status'] : 'Auto';
host['wildcard_text'] = host['wildcard_parent'] ? 'Child' : host['wildcard_text'];
host['wildcard_text_bg'] = 'warning';
if(!host['is_wildcard']){
host['wildcard_text_bg'] = 'success';
@@ -80,6 +91,7 @@
function hostEditOpen(btn, host){
hostEditCancle();
console.log('host:', host)
host = $.scope.hosts.getByKey(host);
host.__jq_$el.addClass('table-warning');
$editHostForm.find('[name=is_wildcard').attr('disabled', true);
@@ -123,15 +135,73 @@
}
};
async function verifyWildcardRequirements(host){
try{
let res = await app.api.get(`dns/domain/${host}`);
return res.results.length === 1;
}catch(error){
return false;
}
}
async function hostMatchWildcard(host){
try{
let res = await app.api.get(`host/lookup/${host}`);
if(res.results && res.results.is_wildcard){
return res.results;
}
}catch(error){
return false;
}
}
$(document).ready(function(){
// Clone the new host form to be used on edit requests.
$editHostForm = $('#addHost').clone();
$editHostForm.find('hr.buttonBreak').nextAll().remove();
// $editHostForm.find('.autoSll').addClass('bg-secondary');
hostPopulate(); //populate the table
// Populate the host UI table
hostPopulate();
// Determine what lets encrypt challenge type the given host name can use
$hostField = $('[name=host');
$hostField.keyup(async function(){
// Reset the allowed types on start
$('#challengeType-child-container').addClass('challengeType-container');
$('#challengeType-DNS-01-wildcard-container').addClass('challengeType-container');
let host = $hostField.val();
// If its a wild card, we must check if the domain has a registered
// provider.
if(host.startsWith("*.") && await verifyWildcardRequirements(host)){
$('#challengeType-DNS-01-wildcard-container').removeClass('challengeType-container');
return;
}
// Check if a wildcard cert is available for the given host.
let wildcardParent = await hostMatchWildcard($hostField.val());
if(wildcardParent){
$('#challengeType-child-container').removeClass('challengeType-container');
$('#challengeType-child-relatedHost').text(wildcardParent.host);
return;
}
// If we hit here, make sure the form is reverted to a valid state
$('#challengeType-child-relatedHost').text('');
$('#challengeType-HTTP-01').prop('checked', true);
});
//
$.scope.hosts.take = function($el, item, list){
$el.addClass('table-danger');
$el.fadeOut(1000, function(){
$el.fadeOut(500, function(){
$el.remove()
});
};
@@ -149,9 +219,9 @@
$el.slideUp();
};
app.subscribe(/^model:Host/, function(data, topic){
console.log(topic, data);
});
// app.subscribe(/^model:Host/, function(data, topic){
// console.log(topic, data);
// });
app.subscribe(/^model:Host:create/, function(data, topic){
let [a,b, action, host] = topic.split(':');
@@ -259,26 +329,8 @@
</div>
</div>
<div class="form-group autoSll">
<label class="form-label">
Auto SSL
</label>
<div class="radio">
<label>
<input type="radio" name="is_wildcard" id="is_wildcard-false" value="false" checked>
On demand certs
</label>
</div>
<div class="radio">
<label>
<input type="radio" name="is_wildcard" id="is_wildcard-true" value="true">
Request Wildcard cert
</label>
</div>
</div>
<div class="form-group">
<label for='host' class="form-label">
<label for="host" class="form-label">
Incoming Host Name
</label>
<div>
@@ -287,6 +339,30 @@
</div>
</div>
<div class="form-group autoSll">
<label class="form-label">
SSL <a href="https://letsencrypt.org/docs/challenge-types/" target="_blank">Validation Type</a>:
</label>
<div class="radio" id="challengeType-HTTP-01-container">
<label>
<input type="radio" name="challengeType" id="challengeType-HTTP-01" value="HTTP-01" checked>
HTTP-01
</label>
</div>
<div class="radio challengeType-container" id="challengeType-DNS-01-wildcard-container">
<label>
<input type="radio" name="challengeType" id="challengeType-DNS-01-wildcard" value="DNS-01-wildcard">
DNS-01 Wildcard
</label>
</div>
<div class="radio challengeType-container" id="challengeType-child-container">
<label>
<input type="radio" name="challengeType" id="challengeType-wildcardChild" value="wildcardChild">
Parent Wildcard from <i id="challengeType-child-relatedHost"></i>
</label>
</div>
</div>
<div class="mb-3 form-group">
<label for="ip" class="form-label">
Target IP or Host Name
@@ -362,6 +438,19 @@
<table class="m-0 card-body table table-striped overflow-x-scroll">
<thead>
<th>
<input type="checkbox"
onclick="$('.host_checkbox:visible').each((i, element)=>{
$(element).prop('checked', $(this).prop('checked'));
})"
/>
<br />
<button type="button" class="btn btn-sm btn-danger" onclick="$('.host_checkbox:checked').each((i, element)=>{
app.api.delete(`host/${$(element).parents('[jq-repeat-index]').attr('jq-repeat-index')}`, function(){});
})">
<i class="fa-solid fa-trash-can"></i>
</button>
</th>
<th>
SSL Expire
</th>
@@ -380,7 +469,10 @@
</thead>
<tbody>
<tr action="api" jq-repeat="hosts" jq-repeat-index='host' style="display:none">
<tr action="api" jq-repeat="hosts" jq-index-key='host' style="display:none">
<td>
<input type="checkbox" class="host_checkbox">
</td>
<td class="table-{{wildcard_text_bg}}">
{{{ wildcard_text }}}
{{#wildcard_expires}}
@@ -395,6 +487,10 @@
<br />
<img width="24px" src="{{displayIconHtml}}" /> {{displayName}} - {{name}}
{{/domain.provider}}
{{#wildcard_parent}}
<i>{{wildcard_parent}}</i>
{{/wildcard_parent}}
</td>
<td>
{{{ targetssl_text }}}{{ ip }}:{{ targetPort }}
+1 -1
View File
@@ -17,7 +17,7 @@
<script type="text/javascript" src="/static-modules/bootstrap/dist/js/bootstrap.bundle.min.js"></script>
<script type="text/javascript" src="/static-modules/@fortawesome/fontawesome-free/js/all.min.js"></script>
<script type="text/javascript" src='/static-modules/mustache/mustache.min.js'></script>
<script type="text/javascript" src='/static/lib/js/jq-repeat_new.js'></script>
<script type="text/javascript" src='/static-modules/jq-repeat/dist/js/jq-repeat.js'></script>
<script type="text/javascript" src='/static/lib/js/val.js'></script>
<script type="text/javascript" src="/static-modules/moment/moment.js"></script>
<script type="text/javascript" src="/static/lib/js/app-base.js"></script>