14 Commits

Author SHA1 Message Date
wmantly 8abc2a65e1 Validate torrent after hashString is resolved
The static Torrent.create override was validating the input data before

calling Transmission, but hashString isn't known until after addUrl.

Move validation to the final createData object so search/fill can queue

torrents without providing a hash up front.

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 15:10:58 -04:00
wmantly a66982a00d Fallback to magnet info-hash when Transmission omits hashString
Transmission addUrl sometimes returns a response without a hashString

(e.g. duplicate or malformed response). Extract the info hash from the

magnet link as a fallback, and log the response if neither works.

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 14:57:19 -04:00
wmantly 51e6817393 Add exponential backoff retry to all downstream calls
New controller/retry.js wraps async calls with infinite retries, capped

at 30s with jitter. Applied to: TPB search, TMDB, Emby, Ollama chat,

Transmission RPC, and the status/status interval calls inherit it via

a Proxy around the Transmission client.

Also handles ollama package returning pre-parsed JSON objects.

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 14:44:41 -04:00
wmantly 803a6db033 Add per-episode fallback for incomplete TV seasons
When a complete-season pack cannot be found for a missing/partial season,

fillMissing now searches each missing SxxExx individually and queues the

best episode torrent per gap. Also exposes episode-level detail in the

/fill response and season plan.

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 14:35:08 -04:00
wmantly ce0e0bdd4f Point Ollama to local host 192.168.1.148:11434
Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 12:49:37 -04:00
wmantly bd9f058f26 Use official ollama npm client for all LLM calls
Replaces hand-rolled fetch to /api/chat with the ollama@0.6.3 package.

The package supplies proper Accept/Content-Type/User-Agent headers and

auto-handles the Ollama protocol, fixing the CDN 403s Node's bare fetch

gets against ollama.com during filing.

Both organize and search LLM ranking now route through controller/ollama.js.

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 12:43:25 -04:00
wmantly 47a930eddb Add explicit User-Agent to Ollama API calls
Node's default fetch sends a bare User-Agent that Google's CDN

fronting ollama.com rejects with 403. curl worked because it

sends curl/. Fixes organize (and search LLM ranking).

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 12:29:09 -04:00
wmantly 40a39b2d14 Add Ollama/organize logging to debug 403 filing failures
- Log raw Ollama HTTP body on non-2xx responses

- Log JSON extraction failures with raw content

- Log organizeWatcher tick errors instead of swallowing them

Signed-off-by: William Mantly <wmantly@gmail.com>
2026-07-20 11:55:41 -04:00
wmantly e9a376925d jq-repeat update 2026-07-19 00:25:04 -04:00
wmantly b166403309 Remember unavailable movies and auto-grab them later (wishlist)
When Smart Search finds no good copy of a movie (unreleased / only CAMs), we
now record it instead of dropping it:
- New Wanted model + migration (dedup by tmdbId, status wanted|fulfilled).
- POST /__api/search/releases remembers a movie when it returns no options,
  and the UI confirms it was added to the wishlist.
- controller/wantedWatcher re-runs Smart Search for wishlisted items every 6h
  (and on boot); when a good copy appears it queues the recommended release,
  which then flows through the normal download + organize pipeline, and marks
  the item fulfilled.
- GET/DELETE /__api/search/wanted to view/remove the wishlist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 00:51:48 -04:00
wmantly 2046e5080b Stop Smart Search recommending the wrong movie / CAMs
TPB search is fuzzy: querying "Toy Story 5" returns Toy Story 4/3/… plus the
only real 2026 copies, which are CAM/Telesync. The LLM rejected them all, then
fallbackReleases picked the most-seeded (Toy Story 4), ignoring title/year.

Two deterministic guards:
- prefilter now drops low-quality source tags (CAM/TS/TELESYNC/SCR/…) via a
  token match, so neither the LLM nor the fallback can surface them.
- findReleases requires the confirmed TMDB year in a movie's release name, so a
  fuzzy match can't cross to a different film. If nothing survives, return no
  options and tell the user it may be unreleased / only CAMs exist.

Verified: "Toy Story 5" now returns 0 options (was Toy Story 4); Night of the
Living Dead (1990) still recommends its 1080p x265 pack.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 22:34:24 -04:00
wmantly 27a6656ac5 Make Smart Search library-aware (Emby dedup + TV gap-fill)
New controller/library.js queries Emby by TMDB provider id (resilient: an
unreachable Emby degrades to "unknown" so search still works).

Movies — quality-aware dedup:
- findReleases checks movieInLibrary(tmdbId) and flags each option alreadyOwned
  when its resolution is <= what you already own, so only genuine upgrades are
  offered; the release picker greys owned qualities and shows a library banner.

TV — library-driven "fill the gaps":
- GET /__api/search/seasons compares Emby's episode inventory against TMDB's
  per-season episode counts to mark each season complete/partial/missing.
- POST /__api/search/fill picks the best complete-season pack (LLM) for every
  missing/incomplete season and queues them. Front-end routes TV titles to a
  season plan + "Fill all gaps" instead of the movie release list.

Adds tmdb.tmdbSeasons; shared tmdb module import in search.js.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 19:11:30 -04:00
wmantly fd5ef14999 Harden auth, add AI smart-search and post-download organization
Security & correctness hardening:
- Gate /__api/token/auth behind auth and self-scope every handler to the
  caller (was fully unauthenticated — account-takeover hole).
- Escape LDAP filter values (injection) and reject empty-password binds.
- Enforce per-torrent ownership so private torrents aren't exposed via IDOR.
- Assorted cleanup: fix 'use static' typos, drop dead Torrent.migrate + the
  getTorrentData noUpdate flag, Buffer.alloc, __dirname-relative reads,
  res.statusCode in the error handler, const-scope pubsub.

Login-gated proxy + anti-indexing:
- Block all proxying for logged-out users via an auth-token cookie the front
  end mirrors from its token; serve a local login page instead of hitting TPB.
- robots.txt disallow-all + X-Robots-Tag noindex.

Torrent category:
- Store a normalized category (TV/Movie/Music/Adult/App/Game/Other) mapped
  from the TPB category id; captured at add time (migration).

Smart Search (movies/TV):
- New /__api/search: TMDB title confirm -> scrape piratebay.party HTML ->
  Ollama ranks releases against quality prefs (x265/1080p/~1.5GB/subs,
  prefer uncut) returning a recommended pick, optional warned 4K, and other
  editions. Front-end Smart Search box + dialog feeding the existing add flow.

Post-download organization -> Emby (public Movie/TV only):
- Completion watcher files finished torrents: Ollama parses the release name,
  TMDB canonicalizes title/year, files main video (+subs) into the library
  with edition/quality-aware names (movies + TV SxxExx), stops seeding, and
  triggers an Emby library scan. Low-confidence matches are flagged, not
  mis-filed; correctable via "Fix match". Adds organizedAt/metadata columns.

Shared helpers: controller/tmdb.js, controller/ollama.js. Config blocks for
tmdb/ollama/search/emby/library/organize (secrets stay in gitignored secrets.js).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:19:13 -04:00
wmantly b30fe748b2 Merge pull request 'private' (#10) from private into master
Reviewed-on: http://git.theta42.com/718itbiz/tpbproxy/pulls/10
2025-11-17 18:35:56 +00:00
30 changed files with 1742 additions and 395 deletions
+3 -3
View File
@@ -17,12 +17,12 @@ const app = express();
// List of front end node modules to be served
const frontEndModules = [
'jquery', 'jquery-ui', 'moment', 'mustache',
'jquery', 'jquery-ui', 'moment', 'mustache', 'jq-repeat',
// 'bootstrap', '@fortawesome',
];
// Hold list of functions to run when the server is ready
app.onListen = [function(){console.log('hello')}];
app.onListen = [];
// Allow the express app to be exported into other files.
module.exports = app;
@@ -104,7 +104,7 @@ app.use(function(err, req, res, next) {
err.status = 422;
}
if(![404, 422].includes(err.status || res.status)){
if(![404, 422].includes(err.status || res.statusCode)){
console.error(err.message);
console.error(err.stack);
console.error('=========================================');
+38
View File
@@ -23,5 +23,43 @@ module.exports = {
username: 'william',
password: '__IN SRECREST FILE__',
statusUpdateInterval: 500,
},
tmdb: {
apiKey: '__IN SRECREST FILE__',
},
ollama: {
url: 'http://192.168.1.148:11434',
apiKey: '__IN SRECREST FILE__',
model: 'gemma4:31b-cloud',
},
search: {
// TPB HTML mirror we scrape search results from( same host the proxy targets).
// apibay's JSON q.php is Cloudflare-gated, so we parse the HTML listing instead.
tpbBase: 'https://piratebay.party',
trackers: [
'udp://tracker.opentrackr.org:1337/announce',
'udp://open.stealth.si:80/announce',
'udp://tracker.torrent.eu.org:451/announce',
'udp://tracker.openbittorrent.com:6969/announce',
'udp://tracker.coppersurfer.tk:6969/announce',
'udp://open.demonii.com:1337/announce',
'udp://exodus.desync.com:6969/announce',
],
},
emby: {
url: 'https://emby.718it.biz',
apiKey: '__IN SRECREST FILE__',
},
library: {
root: '/media/stuff',
movie: 'movies',
tv: 'tv',
},
organize: {
includePrivate: false,
interval: 10000,
},
wanted: {
interval: 21600000, // re-check the wishlist every 6h
}
};
+14
View File
@@ -0,0 +1,14 @@
'use strict';
const conf = require('>/conf');
const { fetchWithRetry } = require('./retry');
// Ask Emby to (re)scan its libraries so a newly filed item gets indexed.
async function refresh(){
let res = await fetchWithRetry(`${conf.emby.url}/Library/Refresh?api_key=${conf.emby.apiKey}`, {
method: 'POST',
}, { label: 'emby refresh' });
return true;
}
module.exports = { refresh };
+2
View File
@@ -4,4 +4,6 @@ module.exports = {
auth: require('./auth'),
pubsub: require('./pubsub'),
torrent: require('./torrent'),
organizeWatcher: require('./organizeWatcher'),
wantedWatcher: require('./wantedWatcher'),
}
+70
View File
@@ -0,0 +1,70 @@
'use strict';
const conf = require('>/conf');
const { fetchJSONWithRetry } = require('./retry');
// Resolution rank so we can compare "what you own" against a candidate release.
function rankFromWidth(w){
w = Number(w) || 0;
if(w >= 3000) return 4; // 4k
if(w >= 1700) return 3; // 1080p
if(w >= 1000) return 2; // 720p
if(w > 0) return 1; // sd
return 0;
}
function rankFromLabel(label){
label = String(label || '').toLowerCase();
if(/2160|4k|uhd/.test(label)) return 4;
if(label.includes('1080')) return 3;
if(label.includes('720')) return 2;
if(label.includes('480')) return 1;
return 0;
}
function labelFromRank(rank){ return ['', '480p', '720p', '1080p', '4k'][rank] || ''; }
async function embyGet(pathAndQuery){
let sep = pathAndQuery.includes('?') ? '&' : '?';
let res = await fetchJSONWithRetry(`${conf.emby.url}${pathAndQuery}${sep}api_key=${conf.emby.apiKey}`, {}, { label: `emby ${pathAndQuery.split('?')[0]}` });
return res;
}
// Is this movie already in the library, and at what best quality? Never throws — an
// unreachable Emby just means "unknown", so search still works.
async function movieInLibrary(tmdbId){
try{
let data = await embyGet(`/Items?Recursive=true&IncludeItemTypes=Movie&AnyProviderIdEquals=tmdb.${tmdbId}&Fields=Width`);
let items = data.Items || [];
if(!items.length) return { owned: false, rank: 0, quality: '' };
let rank = Math.max(...items.map(i => rankFromWidth(i.Width)));
return { owned: true, rank, quality: labelFromRank(rank), count: items.length };
}catch(error){
return { owned: false, rank: 0, quality: '', unknown: true };
}
}
// Which episodes of a series are already present, grouped by season.
// Returns { owned, seriesId, have: { season: Set(episode) }, counts: { season: n } }.
async function seriesInventory(tmdbId){
try{
let s = await embyGet(`/Items?Recursive=true&IncludeItemTypes=Series&AnyProviderIdEquals=tmdb.${tmdbId}&Fields=ProviderIds`);
let series = (s.Items || [])[0];
if(!series) return { owned: false, have: {}, counts: {} };
let ep = await embyGet(`/Shows/${series.Id}/Episodes`);
let have = {};
for(let e of ep.Items || []){
let sn = e.ParentIndexNumber, en = e.IndexNumber;
if(sn == null || en == null || sn === 0) continue; // skip specials
(have[sn] = have[sn] || new Set()).add(en);
}
let counts = {};
for(let sn of Object.keys(have)) counts[sn] = have[sn].size;
return { owned: true, seriesId: series.Id, have, counts };
}catch(error){
return { owned: false, have: {}, counts: {}, unknown: true };
}
}
module.exports = { movieInLibrary, seriesInventory, rankFromWidth, rankFromLabel, labelFromRank };
+50
View File
@@ -0,0 +1,50 @@
'use strict';
const { Ollama } = require('ollama');
const conf = require('>/conf');
// Shared Ollama client. The npm package adds its own Accept / Content-Type / User-Agent
// headers and handles the Ollama chat protocol, which avoids the CDN-level 403s Node's
// bare fetch gets against ollama.com.
const client = new Ollama({
host: conf.ollama.url,
headers: {
'Authorization': `Bearer ${conf.ollama.apiKey}`,
},
});
// Pull the first JSON object out of a model response( tolerates ```json fences / prose).
function extractJSON(content){
let text = String(content).replace(/```json/gi, '').replace(/```/g, '').trim();
let start = text.indexOf('{');
let end = text.lastIndexOf('}');
if(start === -1 || end === -1){
console.error('extractJSON: no JSON object found in:', content);
throw new Error('No JSON in model response');
}
try{
return JSON.parse(text.slice(start, end + 1));
}catch(error){
console.error('extractJSON: parse failed. extracted:', text.slice(start, end + 1));
throw error;
}
}
// One-shot JSON chat against the configured Ollama model.
async function chatJSON(system, user){
let res = await client.chat({
model: conf.ollama.model,
stream: false,
format: 'json',
messages: [
{ role: 'system', content: system },
{ role: 'user', content: user },
],
});
// The ollama package may return message.content as a JSON string when format:'json' is used.
let content = res.message && res.message.content;
if(typeof content === 'string') return extractJSON(content);
return content;
}
module.exports = { client, chatJSON, extractJSON };
+290
View File
@@ -0,0 +1,290 @@
'use strict';
const fs = require('fs');
const path = require('path');
const conf = require('>/conf');
const ollama = require('>/controller/ollama');
const tmdb = require('>/controller/tmdb');
const emby = require('>/controller/emby');
const ps = require('>/controller/pubsub');
const { withRetry } = require('>/controller/retry');
const VIDEO_EXT = new Set(['.mkv', '.mp4', '.avi', '.m4v', '.ts', '.wmv', '.mov']);
const SUB_EXT = new Set(['.srt', '.ass', '.ssa', '.sub', '.vtt', '.idx']);
function ext(name){ return path.extname(name).toLowerCase(); }
function isVideo(name){ return VIDEO_EXT.has(ext(name)) && !/sample/i.test(name); }
function isSub(name){ return SUB_EXT.has(ext(name)); }
// Strip characters that are illegal / awkward in file paths.
function sanitize(text){
return String(text || '').replace(/[\/\\:*?"<>|]/g, '').replace(/\s+/g, ' ').trim();
}
function pad2(n){ return String(n).padStart(2, '0'); }
// Normalize resolution to the library's convention( 2160p/uhd -> 4k).
function normalizeRes(res){
res = String(res || '').toLowerCase();
if(/2160|4k|uhd/.test(res)) return '4k';
let m = res.match(/(480|720|1080)p?/);
return m ? `${m[1]}p` : '';
}
// "Unrated 1080p" style suffix from edition + quality.
function qualitySuffix(meta){
let res = normalizeRes(meta.quality && meta.quality.resolution);
return [sanitize(meta.edition), res].filter(Boolean).join(' ');
}
// --- LLM metadata extraction ----------------------------------------------
function buildExtractPrompt(today){
return [
`You are a media library organizer. Today's date is ${today}.`,
`Given a torrent's raw release name, its coarse type (Movie or TV), and its list of`,
`video file names, extract clean metadata for filing. Return STRICT JSON only:`,
`{"mediaType":"movie"|"tv","title":"clean title, no year/quality/tags",`,
`"year":"YYYY" or "","edition":"Extended|Director's Cut|Unrated|Uncut|Remastered|`,
`Theatrical|IMAX|Redux|" (notable edition if present, else empty),`,
`"quality":{"resolution":"2160p|1080p|720p|480p|","codec":"x265|x264|"},`,
`"episodes":[{"file":"<exact name from the list>","season":<int>,"episode":<int>}]}`,
`Movies: "episodes" is []. TV: one entry per video file, mapping each file to its`,
`season/episode inferred from the file name (SxxExx, 1x02, etc.).`,
`Do not invent data; leave a field empty or [] when unknown.`,
].join('\n');
}
async function llmExtract({ name, category, files }){
let today = new Date().toISOString().slice(0, 10);
let user = JSON.stringify({ name, type: category, files });
return await withRetry(
() => ollama.chatJSON(buildExtractPrompt(today), user),
{ label: 'ollama llmExtract' }
);
}
// Parse + canonicalize. Returns a metadata object, or one carrying `organizeError`
// (so the caller flags the torrent instead of mis-filing it).
async function resolveMeta({ name, category, files }){
let llm;
try{
llm = await llmExtract({ name, category, files });
}catch(error){
return { organizeError: `parse failed: ${error.message}` };
}
let mediaType = llm.mediaType === 'tv' || llm.mediaType === 'movie'
? llm.mediaType
: (category === 'TV' ? 'tv' : 'movie');
if(!llm.title) return { organizeError: `could not determine a title for "${name}"` };
let best = await tmdb.tmdbFindBest(llm.title, llm.year, mediaType);
if(!best) return { organizeError: `no TMDB match for "${llm.title}" (${llm.year || '?'})` };
return {
mediaType,
title: best.title,
year: best.year,
edition: llm.edition || '',
quality: llm.quality || {},
episodes: Array.isArray(llm.episodes) ? llm.episodes : [],
tmdbId: best.tmdbId,
imdbId: best.imdbId,
};
}
// --- Target path building --------------------------------------------------
// Build the destination path for one video file. `episode` is the matching entry from
// meta.episodes (TV only).
function buildTarget(meta, fileName, episode){
let title = sanitize(meta.title);
let folder = `${title} (${meta.year})`;
let suffix = qualitySuffix(meta);
let e = ext(fileName);
if(meta.mediaType === 'tv' && episode){
let dir = path.join(conf.library.root, conf.library.tv, folder, `Season ${pad2(episode.season)}`);
let se = `S${pad2(episode.season)}E${pad2(episode.episode)}`;
let base = `${folder} - ${se}${suffix ? ' - ' + suffix : ''}${e}`;
return path.join(dir, base);
}
let dir = path.join(conf.library.root, conf.library.movie, folder);
let base = `${folder}${suffix ? ' - ' + suffix : ''}${e}`;
return path.join(dir, base);
}
// If the exact target exists, disambiguate with codec then a counter.
function resolveCollision(target, meta){
if(!fs.existsSync(target)) return target;
let dir = path.dirname(target);
let e = path.extname(target);
let baseNoExt = path.basename(target, e);
let codec = meta.quality && meta.quality.codec ? ' ' + sanitize(meta.quality.codec) : '';
let candidate = path.join(dir, `${baseNoExt}${codec}${e}`);
let n = 2;
while(fs.existsSync(candidate)){
candidate = path.join(dir, `${baseNoExt}${codec} (${n})${e}`);
n++;
}
return candidate;
}
// --- Filesystem move -------------------------------------------------------
async function moveFile(src, dst){
await fs.promises.mkdir(path.dirname(dst), { recursive: true });
try{
await fs.promises.rename(src, dst);
}catch(error){
if(error.code === 'EXDEV'){
await fs.promises.copyFile(src, dst);
await fs.promises.unlink(src);
}else{
throw error;
}
}
}
// Subs that belong to a given video( same basename, ignoring a language suffix).
function subsFor(videoName, subFiles, isMovie){
if(isMovie) return subFiles;
let stem = path.basename(videoName, ext(videoName)).toLowerCase();
return subFiles.filter(s => path.basename(s, ext(s)).toLowerCase().startsWith(stem.slice(0, 20)));
}
// --- Orchestration ---------------------------------------------------------
// File a finished torrent into the library, then remove it from Transmission and refresh
// Emby. Returns { organized:true, ... } or { organized:false, error }.
async function fileTorrent(torrent){
const { Torrent } = require('>/models');
let info = (await Torrent.trClient.get(torrent.hashString, ['downloadDir', 'files', 'name'])).torrents[0];
if(!info) return await flag(torrent, 'torrent not found in Transmission');
let downloadDir = info.downloadDir;
let videoFiles = info.files.filter(f => isVideo(f.name));
let subFiles = info.files.filter(f => isSub(f.name)).map(f => f.name);
if(!videoFiles.length) return await flag(torrent, 'no video files in torrent');
let meta = await resolveMeta({
name: torrent.name,
category: torrent.category,
files: videoFiles.map(f => f.name),
});
if(meta.organizeError) return await flag(torrent, meta.organizeError, meta);
let isMovie = meta.mediaType !== 'tv';
// Movie: file only the largest video. TV: file every episode video.
let toFile = isMovie
? [videoFiles.reduce((a, b) => (b.length > a.length ? b : a))]
: videoFiles;
let filed = [];
let libraryPath = null;
for(let vf of toFile){
let episode = isMovie ? null : (meta.episodes.find(e => e.file === vf.name) || null);
if(!isMovie && !episode) continue; // can't place an episode we couldn't map
let dst = resolveCollision(buildTarget(meta, vf.name, episode), meta);
await moveFile(path.join(downloadDir, vf.name), dst);
filed.push({ path: dst, season: episode ? episode.season : null, episode: episode ? episode.episode : null });
libraryPath = path.dirname(dst);
// carry matching sidecar subtitles, renamed to the video's base name
let dstBase = path.basename(dst, ext(dst));
for(let sub of subsFor(vf.name, subFiles, isMovie)){
try{
await moveFile(path.join(downloadDir, sub), path.join(path.dirname(dst), dstBase + ext(sub)));
}catch(error){ /* subs are best-effort */ }
}
}
if(!filed.length) return await flag(torrent, 'could not map any video files to episodes', meta);
// Move + stop seeding: drop the torrent and delete whatever download data is left.
try{ await Torrent.trClient.remove(torrent.hashString, true); }catch(error){ /* already gone */ }
torrent.organizedAt = new Date();
torrent.metadata = { ...meta, libraryPath, files: filed };
await torrent.save();
try{ await emby.refresh(); }catch(error){ console.error('emby refresh failed:', error.message); }
ps.publish('torrent:organized', { hashString: torrent.hashString, libraryPath });
return { organized: true, libraryPath, files: filed };
}
// Re-file an already-organized torrent under a user-chosen TMDB match. Moves the files
// already in the library to their new canonical paths.
async function refileWithMatch(torrent, tmdbId, mediaType){
if(!torrent.metadata || !Array.isArray(torrent.metadata.files) || !torrent.metadata.files.length){
let error = new Error('NothingToRefile');
error.message = 'This torrent has no filed media to re-match';
error.status = 409;
throw error;
}
let details = await tmdb.tmdbDetails(tmdbId, mediaType);
let meta = {
...torrent.metadata,
mediaType,
title: details.title,
year: details.year,
tmdbId,
imdbId: details.imdbId,
};
delete meta.organizeError;
let filed = [];
let libraryPath = null;
for(let entry of torrent.metadata.files){
let old = typeof entry === 'string' ? entry : entry.path;
let episode = (entry && entry.season != null) ? { season: entry.season, episode: entry.episode } : null;
let dst = resolveCollision(buildTarget(meta, old, episode), meta);
if(dst !== old) await moveFile(old, dst);
filed.push({ path: dst, season: episode ? episode.season : null, episode: episode ? episode.episode : null });
libraryPath = path.dirname(dst);
let oldDir = path.dirname(old);
try{ if(oldDir !== libraryPath) await fs.promises.rmdir(oldDir); }catch(error){ /* not empty / gone */ }
}
meta.files = filed;
meta.libraryPath = libraryPath;
torrent.metadata = meta;
torrent.organizedAt = new Date();
await torrent.save();
try{ await emby.refresh(); }catch(error){ console.error('emby refresh failed:', error.message); }
ps.publish('torrent:organized', { hashString: torrent.hashString, libraryPath });
return { organized: true, libraryPath, files: filed };
}
// Record a problem in metadata without setting organizedAt( leaves it flagged for a
// manual "fix match").
async function flag(torrent, message, meta){
console.error(`organize: ${torrent.hashString} skipped — ${message}`);
torrent.metadata = { ...(meta || {}), organizeError: message };
await torrent.save();
return { organized: false, error: message };
}
module.exports = {
fileTorrent,
refileWithMatch,
resolveMeta,
llmExtract,
buildTarget,
qualitySuffix,
normalizeRes,
sanitize,
moveFile,
resolveCollision,
};
+44
View File
@@ -0,0 +1,44 @@
'use strict';
const conf = require('>/conf');
const organize = require('>/controller/organize');
let lock = false;
// Poll unorganized Movie/TV downloads; when Transmission reports one finished, file it.
async function tick(){
if(lock) return;
lock = true;
try{
const { Torrent } = require('>/models');
let where = { organizedAt: null, category: ['Movie', 'TV'] };
if(!conf.organize.includePrivate) where.isPrivate = false;
let pending = await Torrent.findAll({ where });
if(!pending.length) return;
let byHash = {};
for(let t of pending) byHash[t.hashString.toLowerCase()] = t;
let res = await Torrent.trClient.get(pending.map(t => t.hashString), ['hashString', 'percentDone', 'isFinished']);
for(let tor of res.torrents){
if(!(tor.isFinished || tor.percentDone === 1)) continue;
let t = byHash[String(tor.hashString).toLowerCase()];
if(!t) continue;
if(t.metadata && t.metadata.organizeError) continue; // already tried; needs manual fix
await organize.fileTorrent(t);
}
}catch(error){
// Transmission down / transient — just try again next tick.
console.error('organizeWatcher tick failed:', error.name, error.message);
if(error.stack) console.error(error.stack);
}finally{
lock = false;
}
}
setInterval(tick, conf.organize.interval || 10000);
tick(); // reconcile once on boot
module.exports = { tick };
+3 -1
View File
@@ -1,6 +1,8 @@
'use strict';
const {PubSub} = require('p2psub');
ps = new PubSub();
const ps = new PubSub();
module.exports = ps;
+85
View File
@@ -0,0 +1,85 @@
'use strict';
// Retry an async function with exponential backoff up to a max delay.
// No attempt limit: keeps retrying on transient failures forever.
// Transient = network errors, 5xx, 429. 4xx client errors are NOT retried by default.
const TRANSIENT_NETWORK_CODES = new Set([
'ECONNREFUSED', 'ECONNRESET', 'ETIMEDOUT', 'EAI_AGAIN',
'ENOTFOUND', 'EPIPE', 'ERR_SOCKET_TIMEOUT', 'ETIMEOUT',
]);
function isRetryable(error){
// Network / DNS / socket errors
if(error && error.code && TRANSIENT_NETWORK_CODES.has(error.code)) return true;
// HTTP status codes worth retrying
let status = Number(error && error.status) || Number(error && error.status_code);
if(status >= 500 && status < 600) return true;
if(status === 429) return true;
if(status === 502 || status === 503 || status === 504) return true;
// ResponseError from the ollama package carries status_code
if(error && error.name === 'ResponseError' && status >= 500) return true;
return false;
}
async function sleep(ms){
return new Promise(resolve => setTimeout(resolve, ms));
}
async function withRetry(fn, options = {}){
let baseDelay = options.baseDelay || 1000;
let maxDelay = options.maxDelay || 30000;
let multiplier = options.multiplier || 2;
let jitter = options.jitter || 0.2;
let label = options.label || 'operation';
let shouldRetry = options.shouldRetry || isRetryable;
let attempt = 0;
let delay = baseDelay;
while(true){
try{
return await fn();
}catch(error){
attempt++;
if(!shouldRetry(error)){
console.error(`${label} attempt ${attempt} failed permanently:`, error.message);
throw error;
}
console.error(`${label} attempt ${attempt} failed, retrying in ${delay}ms:`, error.message);
let jittered = Math.round(delay * (1 + (Math.random() * 2 - 1) * jitter));
await sleep(jittered);
delay = Math.min(delay * multiplier, maxDelay);
}
}
}
// Convenience: retry a fetch call. On success returns the Response.
async function fetchWithRetry(url, options = {}, retryOptions = {}){
return await withRetry(async () => {
let res = await fetch(url, options);
if(!res.ok){
let error = new Error(`UpstreamError`);
error.message = `Request to ${url.split('?')[0]} failed( ${res.status})`;
error.status = res.status;
throw error;
}
return res;
}, { label: `fetch ${url.split('?')[0]}`, ...retryOptions });
}
// Convenience: retry a fetch call and parse JSON response.
async function fetchJSONWithRetry(url, options = {}, retryOptions = {}){
let res = await fetchWithRetry(url, options, retryOptions);
return await res.json();
}
module.exports = {
withRetry,
fetchWithRetry,
fetchJSONWithRetry,
isRetryable,
};
+393
View File
@@ -0,0 +1,393 @@
'use strict';
const conf = require('>/conf');
const tmdb = require('>/controller/tmdb');
const { tmdbSearch, tmdbDetails } = tmdb;
const library = require('>/controller/library');
const ollama = require('>/controller/ollama');
const { fetchWithRetry, withRetry } = require('>/controller/retry');
// TPB numeric categories that count as Movies / TV( used for the q.php search and to
// keep the LLM from ever seeing non-video results).
const MOVIE_CATS = [201, 202, 207, 209, 211];
const TV_CATS = [205, 208, 212];
function humanSize(bytes){
bytes = Number(bytes) || 0;
let units = ['B', 'KB', 'MB', 'GB', 'TB'];
let i = 0;
while(bytes >= 1024 && i < units.length - 1){
bytes /= 1024;
i++;
}
return `${bytes.toFixed(bytes < 10 && i > 0 ? 1 : 0)} ${units[i]}`;
}
// Turn a raw TPB category id into 'movie' | 'tv' | null.
function videoKind(category){
category = parseInt(category, 10);
if(TV_CATS.includes(category)) return 'tv';
if(MOVIE_CATS.includes(category) || (category >= 200 && category < 300)) return 'movie';
return null;
}
// --- TPB HTML search ------------------------------------------------------
const SIZE_UNITS = { B: 1, KIB: 1024, MIB: 1024 ** 2, GIB: 1024 ** 3, TIB: 1024 ** 4 };
function sizeToBytes(value, unit){
return Math.round(parseFloat(value) * (SIZE_UNITS[String(unit).toUpperCase()] || 1));
}
function decodeEntities(text){
return String(text)
.replace(/&amp;/g, '&').replace(/&lt;/g, '<').replace(/&gt;/g, '>')
.replace(/&quot;/g, '"').replace(/&#0?39;/g, "'").replace(/&apos;/g, "'");
}
// Parse the classic TPB HTML result table( category cell, details link name, magnet,
// then right-aligned size / seeders / leechers cells) into apibay-shaped objects.
function parseTPBHtml(html){
let out = [];
for(let row of html.split(/<tr/i)){
let hash = row.match(/magnet:\?xt=urn:btih:([A-Fa-f0-9]+)/i);
if(!hash) continue;
let cat = row.match(/\/browse\/(\d+)/);
let name = row.match(/title="Details for ([^"]+)"/i);
let size = row.match(/<td align="right">\s*([\d.]+)(?:&nbsp;|\s)+(GiB|MiB|KiB|TiB)\s*<\/td>/i);
let nums = [...row.matchAll(/<td align="right">\s*([\d,]+)\s*<\/td>/gi)].map(m => Number(m[1].replace(/,/g, '')));
if(!name) continue;
// nums are [size, seeders, leechers]; size cell also matched the regex above so
// seeders/leechers are the last two numeric right-aligned cells.
let seeders = nums.length >= 2 ? nums[nums.length - 2] : 0;
let leechers = nums.length >= 1 ? nums[nums.length - 1] : 0;
out.push({
name: decodeEntities(name[1]),
info_hash: hash[1].toLowerCase(),
category: cat ? Number(cat[1]) : 0,
size: size ? sizeToBytes(size[1], size[2]) : 0,
seeders: seeders,
leechers: leechers,
imdb: null,
});
}
return out;
}
async function tpbSearch(title){
// Category 200 = all Video; we narrow to movie/tv subcats in prefilter().
let url = `${conf.search.tpbBase}/search/${encodeURIComponent(title)}/1/99/200`;
let res = await fetchWithRetry(url, { headers: { 'User-Agent': 'Mozilla/5.0' } }, { label: 'tpb search' });
return parseTPBHtml(await res.text());
}
// Low-quality source tags we never want to recommend( token match to avoid false hits
// like "GHOSTS" matching "TS").
const JUNK = new Set(['cam', 'hdcam', 'camrip', 'ts', 'hdts', 'telesync', 'tc', 'telecine', 'scr', 'screener', 'dvdscr', 'workprint']);
function isJunk(name){
return String(name).toLowerCase().split(/[.\s_\-\[\]()]+/).some(tok => JUNK.has(tok));
}
// Deterministic pre-filter before the torrents ever reach the LLM.
function prefilter(torrents, imdbId){
let out = torrents.filter(t => videoKind(t.category) && Number(t.seeders) > 0 && !isJunk(t.name));
// If we can positively identify the title by IMDB id, trust it and drop the rest.
if(imdbId){
let matches = out.filter(t => t.imdb && t.imdb === imdbId);
if(matches.length) out = matches;
}
return out
.sort((a, b) => Number(b.seeders) - Number(a.seeders))
.slice(0, 40);
}
function buildMagnet(infoHash, name){
let magnet = `magnet:?xt=urn:btih:${infoHash}&dn=${encodeURIComponent(name)}`;
for(let tracker of conf.search.trackers){
magnet += `&tr=${encodeURIComponent(tracker)}`;
}
return magnet;
}
// --- LLM ranking ----------------------------------------------------------
function buildSystemPrompt(title, year, today){
return [
`You are a torrent-selection assistant. Today's date is ${today}.`,
`Recent releases are legitimate: NEVER reject a torrent for being too new or assume`,
`a current-year release is fake.`,
``,
`The user wants "${title}" (${year || 'unknown year'}). From the candidate list,`,
`select the best releases.`,
year ? `IMPORTANT: only pick releases whose name contains the year ${year}; reject other years or remakes of a same-named title.` : ``,
`Preferences, in order:`,
`- video codec HEVC/x265 (over x264/AVC)`,
`- 1080p resolution`,
`- total size around 1.5 GB (avoid needlessly huge files)`,
`- English audio and English subtitles present`,
`- prefer EXTENDED / UNRATED / UNCUT / DIRECTOR'S CUT editions`,
`Reject: CAM, TS, TC, TELESYNC, HDCAM, SCREENER/SCR, and non-English-only copies.`,
``,
`Return STRICT JSON only, shape:`,
`{"options":[{"role":"recommended|uhd|other","label":"...","info_hash":"...",`,
`"name":"...","resolution":"...","codec":"...","hasSubs":true,"cut":"...",`,
`"warning":"...","why":"..."}]}`,
`Rules: exactly one "recommended" (the 1080p sweet spot). Include one "uhd" ONLY`,
`if a 4K/2160p release exists, and set its "warning" to note it is a much larger,`,
`slower download. Add "other" entries for genuinely distinct useful releases`,
`(e.g. a different cut or a much smaller copy). "why" is one short sentence.`,
`Copy info_hash verbatim from the chosen candidate.`,
].join('\n');
}
// Pull the first JSON object out of a model response( tolerates ```json fences / prose).
function extractJSON(content){
let text = String(content).replace(/```json/gi, '').replace(/```/g, '').trim();
let start = text.indexOf('{');
let end = text.lastIndexOf('}');
if(start === -1 || end === -1) throw new Error('No JSON in model response');
return JSON.parse(text.slice(start, end + 1));
}
async function rankReleases(candidates, meta){
let compact = candidates.map(t => ({
name: t.name,
sizeHuman: humanSize(t.size),
seeders: Number(t.seeders),
info_hash: t.info_hash,
imdb: t.imdb || null,
}));
let parsed = await withRetry(
() => ollama.chatJSON(
buildSystemPrompt(meta.title, meta.year, meta.today),
JSON.stringify(compact)
),
{ label: 'ollama rankReleases' }
);
let options = Array.isArray(parsed.options) ? parsed.options : [];
if(!options.length) throw new Error('Model returned no options');
return options;
}
// Deterministic fallback if the LLM is unavailable or returns junk: pick the most-seeded
// candidate, preferring x265 + 1080p, so the feature degrades instead of failing.
function fallbackReleases(candidates){
let score = t => (/x265|hevc/i.test(t.name) ? 2 : 0) + (/1080p/i.test(t.name) ? 1 : 0);
let best = [...candidates].sort((a, b) =>
(score(b) - score(a)) || (Number(b.seeders) - Number(a.seeders))
)[0];
if(!best) return [];
return [{
role: 'recommended',
label: 'Most seeded',
info_hash: best.info_hash,
name: best.name,
sizeHuman: humanSize(best.size),
seeders: Number(best.seeders),
why: 'Automatically chosen (AI ranking unavailable): most seeders.',
}];
}
// Attach the fields the front end / add flow need to each option and drop any the model
// hallucinated( info_hash must exist in the candidate set).
function decorateOptions(options, candidates){
let byHash = {};
for(let t of candidates) byHash[String(t.info_hash).toLowerCase()] = t;
let out = [];
for(let opt of options){
let src = byHash[String(opt.info_hash).toLowerCase()];
if(!src) continue;
out.push({
...opt,
info_hash: src.info_hash,
name: opt.name || src.name,
category: Number(src.category),
sizeHuman: opt.sizeHuman || humanSize(src.size),
seeders: Number(src.seeders),
magnetLink: buildMagnet(src.info_hash, src.name),
});
}
return out;
}
async function findReleases(tmdbId, mediaType){
let meta = await tmdbDetails(tmdbId, mediaType);
let torrents = await tpbSearch(meta.title);
let candidates = prefilter(torrents, meta.imdbId);
// For movies, require the confirmed release year in the name. TPB search is fuzzy
// (a "Toy Story 5" query returns Toy Story 4/3/…), and without this the fallback can
// cross to the wrong movie. If nothing matches, we genuinely have no good copy yet.
if(mediaType !== 'tv' && meta.year){
candidates = candidates.filter(t => t.name.includes(meta.year));
}
if(!candidates.length){
return { title: meta.title, year: meta.year, tmdbId, mediaType, options: [] };
}
let today = new Date().toISOString().slice(0, 10);
let options;
try{
options = await rankReleases(candidates, { title: meta.title, year: meta.year, today });
}catch(error){
console.error('rankReleases failed, using fallback:', error.message);
options = fallbackReleases(candidates);
}
options = decorateOptions(options, candidates);
if(!options.length) options = decorateOptions(fallbackReleases(candidates), candidates);
let result = { title: meta.title, year: meta.year, tmdbId, mediaType, options };
// Quality-aware dedup: flag options you already own at equal-or-better quality.
if(mediaType !== 'tv'){
let lib = await library.movieInLibrary(tmdbId);
if(lib.owned){
for(let o of options) o.alreadyOwned = library.rankFromLabel(o.resolution) <= lib.rank;
result.library = { owned: true, quality: lib.quality };
}
}
return result;
}
// --- TV: library-aware season fill -----------------------------------------
function buildSeasonPrompt(title, season, today){
return [
`You are a torrent selector. Today's date is ${today}.`,
`Pick the single best COMPLETE season pack for "${title}" Season ${season}.`,
`It MUST be the full season (an S${String(season).padStart(2, '0')} / "Season ${season}"`,
`complete pack), NOT a single episode. Prefer HEVC/x265, 1080p, English audio +`,
`English subtitles, reasonable size. Reject single episodes, wrong seasons,`,
`CAM/TS/telesync, and non-English-only.`,
`Return STRICT JSON {"info_hash":"<from candidates>","why":"one short sentence"};`,
`if nothing suitable, {"info_hash":null}.`,
].join('\n');
}
async function bestSeasonPack(candidates, meta){
let compact = candidates.map(t => ({ name: t.name, sizeHuman: humanSize(t.size), seeders: Number(t.seeders), info_hash: t.info_hash }));
let parsed = await withRetry(
() => ollama.chatJSON(buildSeasonPrompt(meta.title, meta.season, meta.today), JSON.stringify(compact)),
{ label: 'ollama bestSeasonPack' }
);
return parsed && parsed.info_hash ? parsed : null;
}
// Compare TMDB's season/episode counts against the Emby inventory to classify each season.
async function seasonPlan(tmdbId){
let [tv, inv] = await Promise.all([ tmdb.tmdbSeasons(tmdbId), library.seriesInventory(tmdbId) ]);
let seasons = tv.seasons.map(s => {
let haveSet = inv.have[s.season] || new Set();
let haveCount = haveSet.size;
let status = haveCount === 0 ? 'missing' : (haveCount >= s.episodeCount ? 'complete' : 'partial');
let missingEps = [];
if(status === 'partial'){
for(let e = 1; e <= s.episodeCount; e++){
if(!haveSet.has(e)) missingEps.push(e);
}
}
return { season: s.season, episodeCount: s.episodeCount, haveCount, status, missingEpisodes: missingEps };
});
return { title: tv.title, year: tv.year, seasons, missing: seasons.filter(s => s.status !== 'complete').map(s => s.season) };
}
// Best single episode torrent for a specific SxxExx.
async function bestEpisodeTorrent(candidates, meta){
let compact = candidates.map(t => ({
name: t.name,
sizeHuman: humanSize(t.size),
seeders: Number(t.seeders),
info_hash: t.info_hash,
}));
let prompt = [
`You are a torrent selector. Pick the single best torrent for "${meta.title}"`,
`Season ${meta.season} Episode ${meta.episode}.`,
`It MUST be this exact episode (S${String(meta.season).padStart(2, '0')}E${String(meta.episode).padStart(2, '0')}), not a pack or a different episode.`,
`Prefer HEVC/x265, 1080p, English audio, reasonable size. Reject CAM/TS/telesync and non-English-only.`,
`Return STRICT JSON {"info_hash":"<from candidates>","why":"one short sentence"}; if nothing suitable, {"info_hash":null}.`,
].join(' ');
let parsed = await withRetry(
() => ollama.chatJSON(prompt, JSON.stringify(compact)),
{ label: 'ollama bestEpisodeTorrent' }
);
return parsed && parsed.info_hash ? parsed : null;
}
// For every missing/incomplete season, first try a complete-season pack. If that fails,
// fall back to searching each missing individual episode and queue them separately.
async function fillMissing(tmdbId){
let plan = await seasonPlan(tmdbId);
let details = await tmdbDetails(tmdbId, 'tv');
let title = details.title || plan.title;
let today = new Date().toISOString().slice(0, 10);
let picks = [], skipped = [];
for(let seasonInfo of plan.seasons.filter(s => s.status !== 'complete')){
let season = seasonInfo.season;
// 1) Try a complete season pack first.
let seasonCands = prefilter(await tpbSearch(`${title} S${String(season).padStart(2, '0')}`), null);
if(seasonCands.length){
let pick = null;
try{ pick = await bestSeasonPack(seasonCands, { title, season, today }); }catch(error){ pick = null; }
let src = pick && seasonCands.find(c => c.info_hash.toLowerCase() === String(pick.info_hash).toLowerCase());
if(src){
picks.push({ type: 'season', season, name: src.name, info_hash: src.info_hash, category: Number(src.category), magnetLink: buildMagnet(src.info_hash, src.name), why: pick.why });
continue;
}
}
// 2) No pack. If we know exactly which episodes are missing, grab them one by one.
let missingEps = seasonInfo.missingEpisodes.length ? seasonInfo.missingEpisodes : [];
if(!missingEps.length){
// Fallback: episode-by-episode for the whole season when Emby inventory is unavailable.
for(let e = 1; e <= seasonInfo.episodeCount; e++) missingEps.push(e);
}
let seasonQueued = 0;
for(let episode of missingEps){
let seLabel = `S${String(season).padStart(2, '0')}E${String(episode).padStart(2, '0')}`;
let cands = prefilter(await tpbSearch(`${title} ${seLabel}`), null);
if(!cands.length){ skipped.push({ season, episode, reason: 'no results' }); continue; }
let pick = null;
try{ pick = await bestEpisodeTorrent(cands, { title, season, episode }); }catch(error){ pick = null; }
let src = pick && cands.find(c => c.info_hash.toLowerCase() === String(pick.info_hash).toLowerCase());
if(!src){ skipped.push({ season, episode, reason: 'no suitable episode torrent' }); continue; }
picks.push({ type: 'episode', season, episode, name: src.name, info_hash: src.info_hash, category: Number(src.category), magnetLink: buildMagnet(src.info_hash, src.name), why: pick.why });
seasonQueued++;
}
if(!seasonQueued) skipped.push({ season, reason: 'no suitable season pack or episodes' });
}
return { title, year: plan.year, picks, skipped };
}
module.exports = {
tmdbSearch,
findReleases,
seasonPlan,
fillMissing,
// exported for unit tests
prefilter,
buildMagnet,
humanSize,
videoKind,
extractJSON,
decorateOptions,
fallbackReleases,
tpbSearch,
parseTPBHtml,
sizeToBytes,
isJunk,
};
+75
View File
@@ -0,0 +1,75 @@
'use strict';
const conf = require('>/conf');
const { fetchJSONWithRetry } = require('./retry');
async function tmdbSearch(query){
let url = `https://api.themoviedb.org/3/search/multi?include_adult=false`
+ `&query=${encodeURIComponent(query)}&api_key=${conf.tmdb.apiKey}`;
let data = await fetchJSONWithRetry(url);
return (data.results || [])
.filter(item => item.media_type === 'movie' || item.media_type === 'tv')
.slice(0, 4)
.map(item => {
let date = item.release_date || item.first_air_date || '';
return {
tmdbId: item.id,
mediaType: item.media_type,
title: item.title || item.name,
year: date ? date.slice(0, 4) : '',
posterUrl: item.poster_path ? `https://image.tmdb.org/t/p/w200${item.poster_path}` : null,
};
});
}
async function tmdbDetails(tmdbId, mediaType){
let url = `https://api.themoviedb.org/3/${mediaType}/${tmdbId}`
+ `?append_to_response=external_ids&api_key=${conf.tmdb.apiKey}`;
let data = await fetchJSONWithRetry(url);
let date = data.release_date || data.first_air_date || '';
return {
title: data.title || data.name,
year: date ? date.slice(0, 4) : '',
imdbId: data.imdb_id || (data.external_ids && data.external_ids.imdb_id) || null,
};
}
// Canonicalize a( possibly messy) title + year to the authoritative TMDB record.
// Returns null when nothing matches so callers can flag instead of guessing.
async function tmdbFindBest(title, year, mediaType){
let results = await tmdbSearch(title);
let pool = results.filter(r => r.mediaType === mediaType);
if(!pool.length) pool = results;
if(!pool.length) return null;
let pick = (year && pool.find(r => r.year === String(year))) || pool[0];
let details = await tmdbDetails(pick.tmdbId, pick.mediaType);
return {
title: details.title,
year: details.year,
tmdbId: pick.tmdbId,
mediaType: pick.mediaType,
imdbId: details.imdbId,
};
}
// Per-season episode counts for a show( season 0 / specials excluded).
async function tmdbSeasons(tmdbId){
let data = await fetchJSONWithRetry(`https://api.themoviedb.org/3/tv/${tmdbId}?api_key=${conf.tmdb.apiKey}`);
let date = data.first_air_date || '';
return {
title: data.name,
year: date ? date.slice(0, 4) : '',
seasons: (data.seasons || [])
.filter(s => s.season_number >= 1 && s.episode_count > 0)
.map(s => ({ season: s.season_number, episodeCount: s.episode_count })),
};
}
module.exports = { tmdbSearch, tmdbDetails, tmdbFindBest, tmdbSeasons };
+47
View File
@@ -0,0 +1,47 @@
'use strict';
const conf = require('>/conf');
const search = require('>/controller/search');
let lock = false;
// Periodically re-run Smart Search for wishlisted movies; when a good copy finally
// appears, queue it( which then flows through the normal download + organize pipeline).
async function tick(){
if(lock) return;
lock = true;
try{
const { Wanted, Torrent } = require('>/models');
let wanted = await Wanted.findAll({ where: { status: 'wanted' } });
for(let w of wanted){
if(!w.requestedBy) continue;
try{
let result = await search.findReleases(w.tmdbId, w.mediaType);
let pick = result.options.find(o => o.role === 'recommended' && !o.alreadyOwned)
|| result.options.find(o => !o.alreadyOwned);
if(!pick) continue;
await Torrent.create({
magnetLink: pick.magnetLink,
isPrivate: false,
added_by: w.requestedBy,
category: pick.category,
});
w.status = 'fulfilled';
await w.save();
}catch(error){
// leave it wanted; try again next tick
}
}
}catch(error){
// DB/Transmission transient — retry next tick
}finally{
lock = false;
}
}
setInterval(tick, conf.wanted.interval || 6 * 60 * 60 * 1000);
tick(); // check once on boot
module.exports = { tick };
+2 -1
View File
@@ -3,5 +3,6 @@
module.exports = {
User: require('./ldap/user').User,
AuthToken: require('./sql').AuthToken,
Torrent: require('./sql').Torrent
Torrent: require('./sql').Torrent,
Wanted: require('./sql').Wanted
};
+15 -2
View File
@@ -11,6 +11,12 @@ var userLUR = new LRUCache({
maxAge: 60000,
});
const escapeFilterValue = function(value){
return String(value).replace(/[\0()*\\]/g, function(char){
return '\\' + char.charCodeAt(0).toString(16).padStart(2, '0');
});
}
const user_parse = function(data){
if(data[conf.userNameAttribute]){
data.username = data[conf.userNameAttribute];
@@ -89,7 +95,7 @@ User.get = async function(data, key){
data.searchKey = data.searchKey || key || conf.userNameAttribute;
data.searchValue = data.searchValue || data.uid;
let filter = `(&${conf.userFilter}(${data.searchKey}=${data.searchValue}))`;
let filter = `(&${conf.userFilter}(${escapeFilterValue(data.searchKey)}=${escapeFilterValue(data.searchValue)}))`;
if(userLUR.get(filter)) return userLUR.get(filter);
const client = new Client({
@@ -136,7 +142,14 @@ User.exists = async function(data, key){
User.login = async function(data){
try{
if(!data.password){
let error = new Error('LDAPLoginFailed');
error.name = 'LDAPLoginFailed';
error.message = 'Invalid Credentials, login failed.';
error.status = 401;
throw error;
}
let user = await this.get(data.uid || data[conf.userNameAttribute] || data.username);
const client = new Client({
@@ -0,0 +1,14 @@
'use strict';
/** @type {import('sequelize-cli').Migration} */
module.exports = {
async up(queryInterface, Sequelize) {
await queryInterface.addColumn('Torrents', 'category', {
type: Sequelize.STRING,
allowNull: false,
defaultValue: 'Other'
});
},
async down(queryInterface, Sequelize) {
await queryInterface.removeColumn('Torrents', 'category');
}
};
@@ -0,0 +1,18 @@
'use strict';
/** @type {import('sequelize-cli').Migration} */
module.exports = {
async up(queryInterface, Sequelize) {
await queryInterface.addColumn('Torrents', 'organizedAt', {
type: Sequelize.DATE,
allowNull: true,
});
await queryInterface.addColumn('Torrents', 'metadata', {
type: Sequelize.JSON,
allowNull: true,
});
},
async down(queryInterface, Sequelize) {
await queryInterface.removeColumn('Torrents', 'organizedAt');
await queryInterface.removeColumn('Torrents', 'metadata');
}
};
@@ -0,0 +1,23 @@
'use strict';
/** @type {import('sequelize-cli').Migration} */
module.exports = {
async up(queryInterface, Sequelize) {
await queryInterface.createTable('Wanteds', {
tmdbId: {
type: Sequelize.STRING,
primaryKey: true,
allowNull: false,
},
mediaType: { type: Sequelize.STRING, allowNull: false, defaultValue: 'movie' },
title: { type: Sequelize.STRING },
year: { type: Sequelize.STRING },
status: { type: Sequelize.STRING, allowNull: false, defaultValue: 'wanted' },
requestedBy: { type: Sequelize.STRING },
createdAt: { allowNull: false, type: Sequelize.DATE },
updatedAt: { allowNull: false, type: Sequelize.DATE },
});
},
async down(queryInterface, Sequelize) {
await queryInterface.dropTable('Wanteds');
}
};
+82 -58
View File
@@ -2,18 +2,33 @@
const Transmission = require('transmission-promise');
const conf = require('>/conf');
const { withRetry } = require('>/controller/retry');
const tr_client = new Transmission(conf.transmission)
const tr_client = new Transmission(conf.transmission);
// Wrap the raw Transmission client so every RPC call retries on transient network errors.
const trClient = new Proxy(tr_client, {
get(target, prop){
let value = target[prop];
if(typeof value !== 'function') return value;
return async function(...args){
return await withRetry(
() => value.apply(target, args),
{ label: `transmission ${String(prop)}` }
);
};
}
});
const statusMap = [
'STOPPED', // 0
'CHECK_WAIT', // 1
'CHECK_WAIT', // 1
'CHECK', // 2
'DOWNLOAD_WAIT', // 3
'DOWNLOAD', // 4
'SEED_WAIT', // 5
'SEED', // 6
'ISOLATED', // 7
'DOWNLOAD_WAIT', // 3
'DOWNLOAD', // 4
'SEED_WAIT', // 5
'SEED', // 6
'ISOLATED', // 7
];
module.exports = (sequelize, DataTypes, Model) => {
@@ -27,78 +42,75 @@ module.exports = (sequelize, DataTypes, Model) => {
// define association here
}
static trClient = tr_client;
static trClient = trClient;
// Map a raw TPB category id( e.g. 207) to one of our buckets. TPB groups by the
// hundreds digit; TV shows are the exception pulled out of the Video group.
static categoryFromTPB(id){
id = parseInt(id, 10);
if([205, 208, 212].includes(id)) return 'TV';
return {
1: 'Music',
2: 'Movie',
3: 'App',
4: 'Game',
5: 'Adult',
6: 'Other',
}[Math.floor(id / 100)] || 'Other';
}
static async create(data, ...args){
try{
// let instance = this.build(data);
// console.log('instance', instance)
await this.build(data).validate();
// console.log('validate', val);
data.isPrivate = data.isPrivate === 'true' ? true : false;
let options = {
'download-dir': data.isPrivate ? `${conf.privateDownloadLocation}/${data.added_by}` : undefined,
};
let res = await tr_client.addUrl(data.magnetLink, options);
let res = await trClient.addUrl(data.magnetLink, options);
return await super.create({
// Transmission usually returns hashString, but if it doesn't (duplicate, malformed
// response, etc.), fall back to parsing the info hash from the magnet link.
let hashString = res && (res.hashString || res.id);
if(!hashString){
let match = data.magnetLink.match(/urn:btih:([A-Fa-f0-9]{40})/i);
if(match) hashString = match[1].toLowerCase();
}
if(!hashString){
console.error('Transmission addUrl response missing hashString:', res);
let error = new Error('TorrentCreateError');
error.message = 'Transmission did not return a hash for this torrent';
throw error;
}
let createData = {
magnetLink: data.magnetLink,
hashString: res.hashString,
hashString,
isPrivate: data.isPrivate,
name: res.name,
name: res && res.name,
added_by: data.added_by,
category: this.categoryFromTPB(data.category),
status: 0,
percentDone: 0,
}, args);
};
// Validate only after hashString is populated.
await this.build(createData).validate();
return await super.create(createData, args);
}catch (error){
console.log('Torrent create error', error);
throw error;
}
}
static async migrate(hashString, username){
try{
let exists = await this.findByPk(hashString);
if(exists){
console.log('torrent in DB, skipping')
return {}
}
let res = ( await tr_client.get(hashString, [
"eta", "percentDone", "status", "rateDownload",
"errorString", "hashString", 'name',
'downloadDir',
'addedDate',
'magnetLink',
'files', //array of files
'filesStats', // array of files with status
'isFinished',
'isStalled',
'peers',
'peersConnected', // array of peers,
'sizeWhenDone',
]) ).torrents[0];
// console.log('date:', res.addedDate, new Date(res.addedDate*1000), 'res:', res)
let instance = await this.build({createdAt: new Date(res.addedDate*1000), ...res, added_by: username});
await instance.save();
return {...res, ...instance.dataValues};
}catch(error){
console.error('migrate error', error);
}
}
async getTorrentData(noUpdate){
async getTorrentData(){
try{
if(this.percentDone === 1) return this.dataValues
let res = ( await tr_client.get(this.hashString, [
let res = ( await trClient.get(this.hashString, [
"eta", "percentDone", "status", "rateDownload",
"errorString", "hashString", 'name',
'downloadDir',
@@ -113,8 +125,7 @@ module.exports = (sequelize, DataTypes, Model) => {
]) ).torrents[0];
await this.update(res);
if(noUpdate) await this.save();
return {...res, ...this.dataValues};
}catch(error){
if(error.code === 'ECONNREFUSED'){
@@ -123,7 +134,7 @@ module.exports = (sequelize, DataTypes, Model) => {
throw e
}
// console.error(`Torrent ${this.hashString} getTorrentData error`, error);
throw error;
throw error
}
}
@@ -139,7 +150,7 @@ module.exports = (sequelize, DataTypes, Model) => {
}
async destroy(){
await await this.constructor.trClient.remove(this.hashString, true);
await this.constructor.trClient.remove(this.hashString, true);
return await super.destroy()
}
}
@@ -162,6 +173,11 @@ module.exports = (sequelize, DataTypes, Model) => {
defaultValue: false,
},
name: DataTypes.STRING,
category: {
type: DataTypes.STRING,
allowNull: false,
defaultValue: 'Other',
},
added_by: {
type: DataTypes.STRING,
ldapModel: 'User',
@@ -188,6 +204,14 @@ module.exports = (sequelize, DataTypes, Model) => {
createdAt: {
type: DataTypes.DATE
},
organizedAt: {
type: DataTypes.DATE,
allowNull: true,
},
metadata: {
type: DataTypes.JSON,
allowNull: true,
},
}, {
sequelize,
modelName: 'Torrent',
+32
View File
@@ -0,0 +1,32 @@
'use strict';
module.exports = (sequelize, DataTypes, Model) => {
class Wanted extends Model {
static associate(models) {
}
}
Wanted.init({
tmdbId: {
type: DataTypes.STRING,
primaryKey: true,
allowNull: false,
},
mediaType: {
type: DataTypes.STRING,
allowNull: false,
defaultValue: 'movie',
},
title: DataTypes.STRING,
year: DataTypes.STRING,
status: {
type: DataTypes.STRING,
allowNull: false,
defaultValue: 'wanted', // wanted | fulfilled
},
requestedBy: DataTypes.STRING,
}, {
sequelize,
modelName: 'Wanted',
});
return Wanted;
};
+30
View File
@@ -13,12 +13,14 @@
"express": "^4.18.2",
"extend": "^3.0.2",
"http-proxy-middleware": "^0.20.0",
"jq-repeat": "^2.2.1",
"jquery": "^3.7.1",
"jquery-ui": "^1.13.2",
"ldapts": "^7.0.7",
"lru-native2": "^1.2.6",
"moment": "^2.30.1",
"mustache": "^4.2.0",
"ollama": "^0.6.3",
"p2psub": "^0.1.9",
"sequelize": "^6.35.2",
"sequelize-cli": "^6.6.2",
@@ -2128,6 +2130,19 @@
"node": ">=10"
}
},
"node_modules/jq-repeat": {
"version": "2.2.1",
"resolved": "https://registry.npmjs.org/jq-repeat/-/jq-repeat-2.2.1.tgz",
"integrity": "sha512-1M0jRo7rJKO2mHbeENNjUx9YMOCobUuG4XElKt2NWDc4+j22wrZELqt7Twy95BBEkZQuwDteoPTd1etupZZPtQ==",
"license": "MIT",
"engines": {
"node": ">=14.0.0"
},
"peerDependencies": {
"jquery": ">=3.0.0",
"mustache": ">=4.0.0"
}
},
"node_modules/jquery": {
"version": "3.7.1",
"resolved": "https://registry.npmjs.org/jquery/-/jquery-3.7.1.tgz",
@@ -2919,6 +2934,15 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/ollama": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/ollama/-/ollama-0.6.3.tgz",
"integrity": "sha512-KEWEhIqE5wtfzEIZbDCLH51VFZ6Z3ZSa6sIOg/E/tBV8S51flyqBOXi+bRxlOYKDf8i327zG9eSTb8IJxvm3Zg==",
"license": "MIT",
"dependencies": {
"whatwg-fetch": "^3.6.20"
}
},
"node_modules/on-finished": {
"version": "2.4.1",
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
@@ -4397,6 +4421,12 @@
"node": ">=12"
}
},
"node_modules/whatwg-fetch": {
"version": "3.6.20",
"resolved": "https://registry.npmjs.org/whatwg-fetch/-/whatwg-fetch-3.6.20.tgz",
"integrity": "sha512-EqhiFU6daOA8kpjOWTL0olhVOF3i7OrFzSYiGsEMB8GcXS+RrzauAERX65xMeNWVqxA6HXH2m69Z9LaKKdisfg==",
"license": "MIT"
},
"node_modules/whatwg-url": {
"version": "14.2.0",
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-14.2.0.tgz",
+2
View File
@@ -16,12 +16,14 @@
"express": "^4.18.2",
"extend": "^3.0.2",
"http-proxy-middleware": "^0.20.0",
"jq-repeat": "^2.2.1",
"jquery": "^3.7.1",
"jquery-ui": "^1.13.2",
"ldapts": "^7.0.7",
"lru-native2": "^1.2.6",
"moment": "^2.30.1",
"mustache": "^4.2.0",
"ollama": "^0.6.3",
"p2psub": "^0.1.9",
"sequelize": "^6.35.2",
"sequelize-cli": "^6.6.2",
+17
View File
@@ -168,12 +168,23 @@ app.auth = (function(app) {
var user = {}
function setToken(token){
localStorage.setItem('APIToken', token);
setCookieToken(token);
}
function getToken(){
return localStorage.getItem('APIToken');
}
// The proxy gates page navigation on a cookie( the auth-token header can not ride
// along on a navigation), so mirror the token into a cookie.
function setCookieToken(token){
document.cookie = 'auth-token=' + encodeURIComponent(token) + '; path=/; max-age=31536000; samesite=lax';
}
function clearCookieToken(){
document.cookie = 'auth-token=; path=/; max-age=0; samesite=lax';
}
function isLoggedIn(callback){
if(getToken()){
return app.api.get('user/me', function(error, data){
@@ -198,6 +209,7 @@ app.auth = (function(app) {
function logOut(callback){
callback = callback || app.util.emptyFuction;
localStorage.removeItem('APIToken');
clearCookieToken();
callback();
}
@@ -231,9 +243,14 @@ app.auth = (function(app) {
$( document ).ready( function(){
isLoggedIn(function(error, isLoggedIn){
if(!error && isLoggedIn){
// Refresh the navigation cookie; if this is the logged out gate page,
// reload now that the cookie is set so the real content is proxied.
setCookieToken(getToken());
if(window.__tbpLoginGate) return location.reload();
$('.tbp_proxy_is_authed').show();
$('.tbp_proxy_not_authed').hide();
}else{
clearCookieToken();
$('.tbp_proxy_is_authed').hide();
$('.tbp_proxy_not_authed').show();
}
-302
View File
@@ -1,302 +0,0 @@
(function($, Mustache){
'use strict';
if (!$.scope) {
$.scope = {};
}
var make = function( element ){
//construct array
function makeArray( input , index ){
var result = [];
Object.defineProperty( result, "__repeatId", {
value: repeatId,
writable: true,
enumerable: false,
configurable: true
} );
Object.defineProperty( result, "__rq_template", {
value: '',
writable: true,
enumerable: false,
configurable: true
} );
Object.defineProperty( result, "__jq_index", {
value: index,
writable: true,
enumerable: false,
configurable: true
} );
function removeEmpty(){
if(result.__jq_empty){
result.__jq_empty.remove();
delete result.__jq_empty;
}
}
result.splice = function(inputValue, ...args){
//splice does all the heavy lifting by interacting with the DOM elements.
var toProto = [...args]
var index;
//if a string is submitted as the index, try to match it to index number
if( typeof arguments[0] === 'string' ){
index = this.indexOf( arguments[0] );//set where to start
if ( index === -1 ) {
return [];
}
}else{
index = arguments[0]; //set where to start
}
toProto.unshift(index)
var howMany = arguments[1]; //sets the amount of fields to remove
var args = Array.prototype.slice.call( arguments ); // coverts arguments into array
var toAdd = args.slice(2); // only keeps fields to add to array
// if the starting point is higher then the total index count, start at the end
if( index > this.length ) {
index = this.length;
}
// if the starting point is negative, start form the end of the array, minus the start point
if( index < 0 ) {
index = this.length - Math.abs( index );
}
// if there are things to add, figure out the how many new indexes we need
if( !howMany && howMany !== 0 ) {
howMany = this.length - index;
}
//not sure why i put this here... but it does matter!
if( howMany > this.length - index ) {
howMany = this.length - index;
}
//figure out how many positions we need to shift the current elements
var shift = toAdd.length - howMany;
// figure out how big the new array will be
// var newLength = this.length + shift;
//removes fields from array based on howMany needs to be removed
for( var i = index; i < +index+howMany; i++ ) {
this.__take.apply( this[index].__jq_$el );
// this.__take.apply( $( '.jq-repeat-'+ this.__repeatId +'[jq-repeat-index="'+ ( i + index ) +'"]' ) );
}
//re-factor element index's
for(var i = 0; i < this.length; i++){
if( i >= index){
this[i].__jq_$el.attr( 'jq-repeat-index', i+shift );
}
}
//if there are fields to add to the array, add them
if( toAdd.length > 0 ){
removeEmpty()
//$.each( toAdd, function( key, value ){
for(var I = 0; I < toAdd.length; I++){
//figure out new elements index
var key = I + index;
// apply values to template
var render = Mustache.render( this.__rq_template, {__id:I, ...toAdd[I]} );
//set call name and index keys to DOM element
var $render = $( render ).addClass( 'jq-repeat-'+ this.__repeatId ).attr( 'jq-repeat-index', key );
//if add new elements in proper stop, or after the place holder.
if( key === 0 ){
$( '.jq-repeat-'+ this.__repeatId +'[jq-repeat-index="holder"]' ).after( $render );
}else{
$( '.jq-repeat-'+ this.__repeatId +'[jq-repeat-index="' + ( key -1 ) + '"]' ).after( $render );
}
Object.defineProperty( toAdd[I], "__jq_$el", {
value: $render,
writable: true,
enumerable: false,
configurable: true
} );
//animate element
this.__put.apply($render, [toAdd[I]]);
}
}
//set and return new array
return Array.prototype.splice.apply(this, toProto);
};
result.push = function(){
//add one or more objects to the array
//set the index value, if none is set make it zero
var index = this.length || 0;
//loop each passed object and pass it to slice
for (var i = 0 ; i < arguments.length; ++i) {
this.splice( ( index + i ), 0, arguments[i] );
}
//return new array length
return this.length;
};
result.unshift = function(item){
return this.splice(0, 0, item);
};
result.pop = function(){
//remove and return array element
return this.splice( -1, 1 )[0];
};
result.reverse = function() {
var temp = this.splice( 0 );
Array.prototype.reverse.apply( temp );
for( var i = 0; i < temp.length; i++ ){
this.push( temp[i] );
}
return this;
};
result.shift = function() {
return this.splice( 0, 1 )[0];
};
result.loop = function(){
var temp = this[0];
this.splice( 0,1 );
this.push( temp );
return temp;
};
result.loopUp = function(){
var temp = this[this.length-1];
this.splice( -1, 1 );
this.splice( 0, 0, temp );
return temp;
};
result.indexOf = function( key, value ){
if( !value ){
value = arguments[0];
key = this.__jq_index;
}
for ( var index = 0; index < this.length; ++index ) {
if( this[index][key] === value ){
return index;
}
}
return -1;
};
result.update = function( key, value, update ){
//set variables using sting for index
// If update is called with no index/key, assume its the 0
if(typeof key === 'object'){
if(this[0]){
return this.update(0, key);
}
return this.splice(0, 1, key);
}
if( !update ){
update = arguments[1];
value = arguments[0];
key = this.__jq_index;
}
var index = this.indexOf( key, value );
if(index === -1) {
return [];
}
var object = $.extend( true, {}, this[index], update );
return this.splice( index, 1, object )[0];
};
result.__put = function(){
this.show();
};
result.__take = function(){
this.remove();
};
if(!input) {
return result;
}
$.each( input, function( key, value ){
var type = typeof value;
if( type === 'object' ){
result.push( value );
}else if( type === 'string' ){
Object.defineProperty( result, "__jq_index", {
value: value,
writable: true,
enumerable: false,
configurable: true
} );
} else if ( type === 'function'){
Object.defineProperty( result, value.name, {
value: value,
writable: true,
enumerable: false,
configurable: true
} );
}
} );
return result;
}
var $this = $( element );
var repeatId = $this.attr( 'jq-repeat' );
var index = $this.attr( 'jq-repeat-index' );
var tempId = repeatId + 'Template';
var templateId = $( '#' + tempId ).html();
var empty = $(`[jq-repeat-defualt="${repeatId}"]`);
$this.removeAttr( 'jq-repeat' );
$this.removeAttr( 'jq-repeat-index' );
var template = element.outerHTML
$this.replaceWith( '<script type="x-tmpl-mustache" id="' + tempId + '" class="jq-repeat-' + repeatId + ' " jq-repeat-index="holder"><\/script>' );
Mustache.parse(templateId); // optional, speeds up future uses
$.scope[repeatId] = makeArray($.scope[repeatId], index);
$.scope[repeatId].__rq_template = template;
$.scope[repeatId].__jq_empty = empty;
};
$( document ).ready( function(){
$( '[jq-repeat]' ).each(function(key, value){
make(value);
});
$(document).on('DOMNodeInserted', function(e) {
if ( $(e.target).is('[jq-repeat]') ){
make( e.target );
}else{
var t = $(e.target).find('[jq-repeat]');
t.each(function(key, value){
make(value);
});
}
});
} );
})(jQuery, Mustache);
+185 -3
View File
@@ -2,7 +2,7 @@
<script type="text/javascript" src="/socket.io/socket.io.js"></script>
<script src='/__static-modules/mustache/mustache.min.js'></script>
<script src="/__static/lib/js/jq-repeat.js"></script>
<script src="/__static-modules/jq-repeat/dist/js/jq-repeat.js"></script>
<script src="/__static-modules/moment/min/moment-with-locales.min.js"></script>
<script src="/__static-modules/jquery-ui/dist/jquery-ui.min.js"></script>
<script src="/__static/js/app.js"></script>
@@ -185,6 +185,15 @@
<p>
Done! <a href="https://stuff.718it.biz/torrents/{{name}}" target="_blank"> HTTP Link</a>
</p>
{{#organized}}
<p>📁 Filed to <b>{{libraryPath}}</b></p>
{{/organized}}
{{#organizeError}}
<p style="color:#b00">⚠ Not filed: {{organizeError}}</p>
{{/organizeError}}
<button class="ui-button ui-widget ui-corner-all" onclick="tbpFixMatch('{{hashString}}')">
<span class="ui-icon ui-icon-pencil"></span> Fix match
</button>
{{/isFinished}}
<hr />
</li>
@@ -241,6 +250,8 @@
<input type="text" name="hashString" value="{{{hashString}}}" readonly/>
</p>
<input type="hidden" name="category" value="{{{category}}}"/>
<p>
<label for="isPrivate-false" title="The download will appare in the communal download folder">Public:</label>
<input type="radio" name="isPrivate" id="isPrivate-false" value="false" />
@@ -268,11 +279,20 @@
</div>
<!--
<div id="tbp_proxy_search_dialog" title="Smart Search">
<div id="tbp_proxy_search_dialog_body"></div>
</div>
<!--
Injected Header bar
-->
<div id="tbp_proxy_header_right">
<form id="tbp_proxy_search_form" class="tbp_proxy_is_authed" style="display:inline-block; margin-right:.5em;" onsubmit="return false;">
<input type="text" id="tbp_proxy_search_input" placeholder="Smart search movies &amp; TV…" />
<button type="submit" class="ui-button ui-corner-all ui-widget">Search</button>
</form>
<span id="tbp_proxy_torrent_dialog_opener" class="tbp_proxy_is_authed">
<img src="/__static/img/Transmission_Icon.svg" height="22" width="22" style="margin-right: .3em;" />
<span jq-repeat="tbp_proxy_torrent_dialog_opener_status">
@@ -320,6 +340,159 @@
$('#tbp_proxy_torrent_add_dialog').dialog(commonDialogOptions);
/* Smart Search button and dialog */
$('#tbp_proxy_search_dialog').dialog(commonDialogOptions);
// Escape untrusted text before dropping it into HTML.
function tbpEsc(text){ return $('<i>').text(text == null ? '' : text).html(); }
function tbpSearchBody(html){ $('#tbp_proxy_search_dialog_body').html(html); }
// Step 1: render the TMDB title candidates to confirm which one they meant.
// onPick defaults to the search flow; the "Fix match" flow passes its own.
function tbpRenderTitles(results, onPick){
onPick = onPick || tbpFindReleases;
if(!results.length){ tbpSearchBody('<p>No matches found.</p>'); return; }
tbpSearchBody('<p>Which one did you mean?</p>');
results.forEach(function(r){
var poster = r.posterUrl ? '<img src="'+ r.posterUrl +'" width="60" style="vertical-align:middle;margin-right:.5em;"/>' : '';
$('<div class="tbp_search_card" style="cursor:pointer;padding:.4em;border-bottom:1px solid #ccc;"></div>')
.html(poster +'<b>'+ tbpEsc(r.title) +'</b> ('+ tbpEsc(r.year || '?') +') <span style="color:#888">'+ tbpEsc(r.mediaType) +'</span>')
.on('click', function(){ onPick(r); })
.appendTo('#tbp_proxy_search_dialog_body');
});
}
// "Fix match": re-run the title picker for a finished torrent and re-file it under
// the chosen TMDB match via POST /torrent/:hash/organize/match.
window.tbpFixMatch = function(hash){
var item = ($.scope.tbp_proxy_torrent_dialog_torrents || []).filter(function(t){ return t.hashString === hash; })[0];
var query = (item ? item.name : '').replace(/[._]/g, ' ').replace(/\b(19|20)\d\d\b.*/, function(m){ return m.slice(0, 4); }).trim();
openDialog($('#tbp_proxy_search_dialog'));
tbpSearchBody('<p>Searching for a better match…</p>');
app.api.get('search/title?q='+ encodeURIComponent(query || (item ? item.name : '')), function(error, data){
if(error || !data || !data.results){ tbpSearchBody('<p>Search failed. Please try again.</p>'); return; }
tbpRenderTitles(data.results, function(r){
tbpSearchBody('<p>Re-filing as <b>'+ tbpEsc(r.title) +' ('+ tbpEsc(r.year) +')</b>…</p>');
app.api.post('torrent/'+ hash +'/organize/match', { tmdbId: r.tmdbId, mediaType: r.mediaType }, function(err, res){
if(err || !res || res.organized !== true){
tbpSearchBody('<p>Re-file failed: '+ tbpEsc((res && res.message) || err) +'</p>'); return;
}
tbpSearchBody('<p>✓ Re-filed to <b>'+ tbpEsc(res.libraryPath) +'</b>. Emby will re-index.</p>');
});
});
});
};
// Step 2: ask the server( TPB + LLM) for the curated release options.
function tbpFindReleases(title){
if(title.mediaType === 'tv') return tbpSeasonPlan(title);
tbpSearchBody('<p>Finding the best copies for <b>'+ tbpEsc(title.title) +'</b>…<br/>this can take a few seconds.</p>');
app.api.post('search/releases', { tmdbId: title.tmdbId, mediaType: title.mediaType }, function(error, data){
if(error || !data || !data.options){ tbpSearchBody('<p>Search failed. Please try again.</p>'); return; }
tbpRenderReleases(data);
});
}
// Step 3: render the release cards; a tap feeds the existing add dialog.
function tbpRenderReleases(data){
if(!data.options.length){
var msg = '<p>No good-quality copy of <b>'+ tbpEsc(data.title) +'</b> found — it may not be released yet, or only low-quality (CAM) copies exist.</p>';
if(data.remembered) msg += '<p style="color:#2a2">✓ Added to your wishlist — we\'ll grab it automatically when a good copy appears.</p>';
tbpSearchBody(msg);
return;
}
var header = '<h3>'+ tbpEsc(data.title +' ('+ (data.year || '?') +')') +'</h3>';
if(data.library && data.library.owned) header += '<p style="color:#2a2">✓ Already in your library ('+ tbpEsc(data.library.quality) +') — only upgrades are offered.</p>';
tbpSearchBody(header);
data.options.forEach(function(o){
if(o.alreadyOwned){
$('<div style="padding:.5em;border-bottom:1px solid #ccc;color:#999;"></div>')
.html('<b>'+ tbpEsc(o.label || o.role) +'</b> — you already have this quality or better'
+ '<div style="font-size:.9em">'+ tbpEsc(o.name) +'</div>')
.appendTo('#tbp_proxy_search_dialog_body');
return;
}
var warn = o.warning ? '<div style="color:#b00;font-weight:bold;">⚠ '+ tbpEsc(o.warning) +'</div>' : '';
var why = o.why ? '<div style="color:#555;font-style:italic;">'+ tbpEsc(o.why) +'</div>' : '';
$('<div class="tbp_search_card" style="cursor:pointer;padding:.5em;border-bottom:1px solid #ccc;"></div>')
.html('<b>'+ tbpEsc(o.label || o.role) +'</b> <span style="color:#888">'+ tbpEsc(o.sizeHuman || '') +' · '+ tbpEsc(o.seeders || 0) +' seeders</span>'
+ '<div style="font-size:.9em">'+ tbpEsc(o.name) +'</div>' + why + warn)
.on('click', function(){ tbpAddRelease(o); })
.appendTo('#tbp_proxy_search_dialog_body');
});
}
// Reuse the existing add flow: fill the torrentAdd scope and open the add dialog.
function tbpSeasonPlan(title){
tbpSearchBody('<p>Checking your library for <b>'+ tbpEsc(title.title) +'</b>…</p>');
app.api.get('search/seasons?tmdbId='+ encodeURIComponent(title.tmdbId), function(error, data){
if(error || !data || !data.seasons){ tbpSearchBody('<p>Could not load seasons.</p>'); return; }
tbpRenderSeasons(title, data);
});
}
function tbpRenderSeasons(title, data){
tbpSearchBody('<h3>'+ tbpEsc(data.title +' ('+ (data.year || '?') +')') +'</h3>');
data.seasons.forEach(function(s){
var badge, color;
if(s.status === 'complete'){ badge = '✓ complete'; color = '#2a2'; }
else if(s.status === 'partial'){ badge = '⚠ '+ s.haveCount +'/'+ s.episodeCount; color = '#b80'; }
else { badge = '✗ missing'; color = '#b00'; }
$('<div style="padding:.3em;border-bottom:1px solid #eee;"></div>')
.html('<b>Season '+ s.season +'</b> <span style="color:'+ color +'">'+ badge +'</span>')
.appendTo('#tbp_proxy_search_dialog_body');
});
if(data.missing && data.missing.length){
$('<button class="ui-button ui-corner-all ui-widget" style="margin-top:.6em;">Fill all gaps ('+ data.missing.length +' season'+ (data.missing.length > 1 ? 's' : '') +')</button>')
.on('click', function(){ tbpFillGaps(title); })
.appendTo('#tbp_proxy_search_dialog_body');
}else{
$('#tbp_proxy_search_dialog_body').append('<p style="color:#2a2">✓ You already have every season.</p>');
}
}
function tbpFillGaps(title){
tbpSearchBody('<p>Finding the best packs for the missing seasons of <b>'+ tbpEsc(title.title) +'</b>…<br/>this can take a bit.</p>');
app.api.post('search/fill', { tmdbId: title.tmdbId }, function(error, data){
if(error || !data){ tbpSearchBody('<p>Fill failed. Please try again.</p>'); return; }
var html = '<h3>'+ tbpEsc(data.title) +'</h3>';
(data.queued || []).forEach(function(q){ html += '<p style="color:#2a2">✓ Queued Season '+ q.season +': '+ tbpEsc(q.name) +'</p>'; });
(data.skipped || []).forEach(function(s){ html += '<p style="color:#b00">✗ Season '+ s.season +': '+ tbpEsc(s.reason) +'</p>'; });
if(!(data.queued || []).length && !(data.skipped || []).length) html += '<p>Nothing to fill.</p>';
tbpSearchBody(html);
});
}
function tbpAddRelease(o){
$.scope.torrentAdd.update({
magnetLink: o.magnetLink,
name: o.name,
hashString: String(o.info_hash).toLowerCase(),
category: o.category,
});
if(localStorage.getItem('isPrivate') === 'true'){
$('#isPrivate-true').prop('checked', true);
}else{
$('#isPrivate-false').prop('checked', true);
}
$('#tbp_proxy_search_dialog').dialog('close');
openDialog($('#tbp_proxy_torrent_add_dialog'));
}
$('#tbp_proxy_search_form').on('submit', function(){
var q = $('#tbp_proxy_search_input').val();
if(!q) return false;
openDialog($('#tbp_proxy_search_dialog'));
tbpSearchBody('<p>Searching…</p>');
app.api.get('search/title?q='+ encodeURIComponent(q), function(error, data){
if(error || !data || !data.results){ tbpSearchBody('<p>Search failed. Please try again.</p>'); return; }
tbpRenderTitles(data.results);
});
return false;
});
/* Enable tooltips*/
$('#tbp_proxy_header').tooltip({
track: true
@@ -332,10 +505,17 @@
// magnetLink
let magnetLinkParams = new URLSearchParams($(this).data('link'));
// Grab the TPB category id from the /browse/<id> link in this
// torrent's row( listing) or the Type: field( detail page).
let $cat = $(this).closest('tr').find('a[href*="/browse/"]').first();
if(!$cat.length) $cat = $('dd a[href*="/browse/"]').first();
let category = $cat.length ? $cat.attr('href').replace(/.*\/browse\//, '').replace(/\D.*$/, '') : '';
$.scope.torrentAdd.update({
magnetLink: $(this).data('link'),
name: magnetLinkParams.get('dn'),
hashString: magnetLinkParams.get('magnet:?xt').split(':').pop().toLowerCase(),
category: category,
});
if(localStorage.getItem('isPrivate') === 'true'){
@@ -521,7 +701,9 @@
"isActive": [3, 4, 5, 6].includes(torrent.status), // DOWNLOAD_WAIT ,DOWNLOAD, SEED_WAIT, SEED
"isFinished": torrent.isFinished || percentDone === 100,
"createdAtString": moment(torrent.createdAt).fromNow(),
"organized": !!torrent.organizedAt,
"libraryPath": torrent.metadata && torrent.metadata.libraryPath,
"organizeError": torrent.metadata && torrent.metadata.organizeError,
}
}
+2 -1
View File
@@ -4,8 +4,9 @@ const router = require('express').Router();
const middleware = require('>/middleware/auth');
router.use('/auth', require('./auth'));
router.use('/token/auth', require('./authtoken'));
router.use('/token/auth', middleware.auth, require('./authtoken'));
router.use('/torrent', middleware.auth, require('./transmission'));
router.use('/search', middleware.auth, require('./search'));
router.use('/user', middleware.auth, require('./user'));
module.exports = router;
+25 -5
View File
@@ -3,10 +3,22 @@
const router = require('express').Router();
const {AuthToken} = require('>/models');
function ownToken(token, req){
if(!token || token.username !== req.user.username){
let error = new Error('AuthTokenNotFound');
error.name = 'AuthTokenNotFound';
error.message = 'Token not found';
error.status = 404;
throw error;
}
return token;
}
router.get('/', async function(req, res, next){
try{
return res.json(await AuthToken.findAll());
return res.json(await AuthToken.findAll({where:{
username: req.user.username
}}));
}catch(error){
next(error);
}
@@ -14,7 +26,7 @@ router.get('/', async function(req, res, next){
router.post('/', async function(req, res, next){
try{
return res.json(await AuthToken.create(req.body));
return res.json(await AuthToken.create({...req.body, username: req.user.username}));
}catch(error){
console.error(error)
next(error);
@@ -23,6 +35,14 @@ router.post('/', async function(req, res, next){
router.get('/user/:username', async function(req, res, next){
try{
if(req.params.username !== req.user.username){
let error = new Error('AuthTokenNotFound');
error.name = 'AuthTokenNotFound';
error.message = 'Token not found';
error.status = 404;
throw error;
}
return res.json(await AuthToken.findAll({where:{
username: req.params.username
}}));
@@ -33,7 +53,7 @@ router.get('/user/:username', async function(req, res, next){
router.get('/:token', async function(req, res, next){
try{
let token = await AuthToken.findByPk(req.params.token)
let token = ownToken(await AuthToken.findByPk(req.params.token), req);
token.dataValues.user = await token.getUser()
return res.json(token);
@@ -44,7 +64,7 @@ router.get('/:token', async function(req, res, next){
router.put('/:token', async function(req, res, next){
try{
let token = await AuthToken.findByPk(req.params.token);
let token = ownToken(await AuthToken.findByPk(req.params.token), req);
await token.update(req.body);
return res.json(token);
}catch(error){
@@ -54,7 +74,7 @@ router.put('/:token', async function(req, res, next){
router.delete('/:token', async function(req, res, next){
try{
let token = await AuthToken.findByPk(req.params.token);
let token = ownToken(await AuthToken.findByPk(req.params.token), req);
await token.destroy();
return res.json({'deleted': true});
+43 -5
View File
@@ -1,19 +1,57 @@
'use static';
'use strict';
const router = require('express').Router();
const path = require('path');
const zlib = require('zlib');
const fs = require('fs');
const https = require('https');
const http = require("http");
const proxy = require('http-proxy-middleware');
const { Auth } = require('>/controller/auth');
const inject = fs.readFileSync('./inject.html', 'utf8');
const mainjs = fs.readFileSync('./static/main.js', 'utf8');
const inject = fs.readFileSync(path.join(__dirname, '..', 'inject.html'), 'utf8');
const mainjs = fs.readFileSync(path.join(__dirname, '..', 'static', 'main.js'), 'utf8');
// Page served to users who are not logged in. It reuses the injected front end
// (jQuery + header partial) so the login dialog is available, but nothing is proxied.
const loginPage = "<html><head><meta name='robots' content='noindex, nofollow'><title>Login</title></head><body>"
+ inject
+ "<script>window.__tbpLoginGate = true;</script>"
+ "</body></html>";
function parseCookies(req){
let out = {};
let header = req.headers.cookie;
if(!header) return out;
for(let pair of header.split(';')){
let idx = pair.indexOf('=');
if(idx < 0) continue;
out[pair.slice(0, idx).trim()] = decodeURIComponent(pair.slice(idx + 1).trim());
}
return out;
}
// Keep the proxy( and everything it serves) out of search engines.
router.get('/robots.txt', function(req, res){
res.type('text/plain').send('User-agent: *\nDisallow: /\n');
});
// Block all proxying for users who are not logged in. Page navigation can not send
// the auth-token header, so the token is read from a cookie set by the front end.
router.use(async function(req, res, next){
res.set('X-Robots-Tag', 'noindex, nofollow');
try{
await Auth.checkToken(parseCookies(req)['auth-token']);
return next();
}catch(error){
return res.status(401).send(loginPage);
}
});
// app.all("/*.js", function(req, res){res.send('')});
router.all('/static/main.js', function(req,res){
res.write(mainjs);
res.end(mainjs);
});
const proxyTarget = {
@@ -70,7 +108,7 @@ router.all("/*", proxy({
// res.set(key, proxyRes.headers[key].toString().replace('http://', 'https://'))
}
let body = new Buffer('');
let body = Buffer.alloc(0);
proxyRes.on('error', function(e){
console.error('ERROR!', e)
});
+92
View File
@@ -0,0 +1,92 @@
'use strict';
const router = require('express').Router();
const search = require('>/controller/search');
const { Torrent, Wanted } = require('>/models');
// Step 1: fuzzy query -> a few TMDB title candidates( with posters) to confirm.
router.get('/title', async function(req, res, next){
try{
res.json({results: await search.tmdbSearch(req.query.q)});
}catch(error){
next(error);
}
});
// Step 2: confirmed title -> TPB search + LLM-curated release options.
router.post('/releases', async function(req, res, next){
try{
let result = await search.findReleases(req.body.tmdbId, req.body.mediaType);
// Nothing good yet( e.g. unreleased) -> remember it and grab it later.
if(!result.options.length && result.mediaType !== 'tv' && result.tmdbId){
await Wanted.upsert({
tmdbId: String(result.tmdbId),
mediaType: result.mediaType,
title: result.title,
year: result.year,
status: 'wanted',
requestedBy: req.user.username,
});
result.remembered = true;
}
res.json(result);
}catch(error){
next(error);
}
});
// Wishlist: things we couldn't find yet and are watching for.
router.get('/wanted', async function(req, res, next){
try{
res.json(await Wanted.findAll({ order: [['createdAt', 'DESC']] }));
}catch(error){
next(error);
}
});
router.delete('/wanted/:tmdbId', async function(req, res, next){
try{
let wanted = await Wanted.findByPk(req.params.tmdbId);
if(wanted) await wanted.destroy();
res.json({ deleted: true });
}catch(error){
next(error);
}
});
// TV: which seasons you have vs are missing (Emby inventory vs TMDB counts).
router.get('/seasons', async function(req, res, next){
try{
res.json(await search.seasonPlan(req.query.tmdbId));
}catch(error){
next(error);
}
});
// TV: queue the best complete-season pack for every missing/incomplete season.
router.post('/fill', async function(req, res, next){
try{
let plan = await search.fillMissing(req.body.tmdbId);
let queued = [];
for(let pick of plan.picks){
try{
await Torrent.create({
magnetLink: pick.magnetLink,
isPrivate: false,
added_by: req.user.username,
category: pick.category,
});
queued.push({ type: pick.type, season: pick.season, episode: pick.episode, name: pick.name });
}catch(error){
plan.skipped.push({ season: pick.season, episode: pick.episode, reason: error.message });
}
}
res.json({ title: plan.title, year: plan.year, queued, skipped: plan.skipped });
}catch(error){
next(error);
}
});
module.exports = router;
+46 -14
View File
@@ -1,12 +1,28 @@
'use static';
'use strict';
const router = require('express').Router();
const {Torrent} = require('>/models');
const organize = require('>/controller/organize');
function authTorrent(torrent, req){
if(torrent && torrent.isPrivate && torrent.added_by !== req.user.username){
let error = new Error('TorrentNotFound');
error.name = 'TorrentNotFound';
error.message = 'Torrent not found';
error.status = 404;
throw error;
}
return torrent;
}
router.get('/', async function(req, res, next){
try{
let username = req.query.username || req.user.username;
let where = {added_by: username};
if(username !== req.user.username) where.isPrivate = false;
res.json({results: await Torrent.findAll({
where:{added_by: req.query.username || req.user.username},
where,
limit: req.query.limit,
offset: req.query.offset,
order: [
@@ -26,14 +42,6 @@ router.post("/", async function(req, res, next){
}
});
router.post("/:hashString", async function(req, res, next){
try{
res.json(await Torrent.migrate(req.params.hashString, req.user.username))
}catch(error){
next(error);
}
});
router.get('/server', async function(req, res, next){
try{
res.json(await Torrent.trClient.sessionStats())
@@ -44,7 +52,7 @@ router.get('/server', async function(req, res, next){
router.get("/:hashString", async function(req, res, next){
try{
let torrent = await Torrent.findByPk(req.params.hashString);
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
if('latest' in req.query){
torrent = await torrent.getTorrentData();
}
@@ -56,7 +64,7 @@ router.get("/:hashString", async function(req, res, next){
router.delete("/:hashString", async function(req, res, next){
try{
let torrent = await Torrent.findByPk(req.params.hashString);
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
res.json({result: torrent, activity: await torrent.destroy()});
}catch(error){
@@ -66,7 +74,7 @@ router.delete("/:hashString", async function(req, res, next){
router.post("/:hashString/stop", async function(req, res, next){
try{
let torrent = await Torrent.findByPk(req.params.hashString);
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
res.json({result: torrent, activity: await torrent.stop()});
}catch(error){
@@ -76,7 +84,7 @@ router.post("/:hashString/stop", async function(req, res, next){
router.post("/:hashString/start", async function(req, res, next){
try{
let torrent = await Torrent.findByPk(req.params.hashString);
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
res.json({result: torrent, activity: await torrent.start()});
}catch(error){
@@ -84,6 +92,30 @@ router.post("/:hashString/start", async function(req, res, next){
}
});
// Manually (re)run organization for a finished torrent.
router.post("/:hashString/organize", async function(req, res, next){
try{
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
if(!torrent){ let e = new Error('TorrentNotFound'); e.status = 404; throw e; }
res.json(await organize.fileTorrent(torrent));
}catch(error){
next(error);
}
});
// Correct a wrong match: re-file an already-organized torrent under a chosen TMDB id.
router.post("/:hashString/organize/match", async function(req, res, next){
try{
let torrent = authTorrent(await Torrent.findByPk(req.params.hashString), req);
if(!torrent){ let e = new Error('TorrentNotFound'); e.status = 404; throw e; }
res.json(await organize.refileWithMatch(torrent, req.body.tmdbId, req.body.mediaType));
}catch(error){
next(error);
}
});
module.exports = router;