sophie/simpleplayer-helper
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Add the run loop: probe every unprobed file once, with backoff #5

Merged
sophie merged 5 commits from run-loop into main 2026-10-16 22:31:03 +11:00
Conversation 0 Commits 5 Files changed 12 +1115 -56
sophie commented 2026-10-16 22:30:57 +11:00
Owner
Copy link

Finishes the helper: it now runs until stopped, probes each file version once and reports the result to the server.

  • HelperRunner:
    • On start and then every POLL_INTERVAL_SECONDS, it fetches GET /media/unprobed until nothing new is listed. For each file it gets a download URL, runs ffprobe and PUTs the result.
    • Transient failures back off per media id and etag (1 min doubling to 1 h, held in memory). If a PUT fails transiently, the probe result is kept and resent without running ffprobe again.
    • 409 and 404 drop the file; 400/422 are logged at error and backed off.
    • If the server is unreachable, it logs one warning, backs off the same way and never exits.
  • main wires up the config, client, prober and runner. SIGTERM/SIGINT kill a running ffprobe and exit 0; only a bad configuration exits non-zero.
  • Logging: one Probed line per file (key, container, duration, resolution, hdr), a warning with the attempt count for each transient failure, and a Pass finished summary per pass (logged at debug when the pass did nothing).
  • Prober fixes:
    • hdr is now true/false for every probed video stream, as the contract says.
    • It waits for ffprobe with a termination handler instead of Process.waitUntilExit(), which sometimes never returned and hung the helper for good.
    • It closes ffprobe's pipes; each run used to leak two file descriptors.
  • Tests: the runner runs against a fake server on the stub, with a scripted prober and a fake clock. There's also a regression test that runs ffprobe 200 times in a row (it hangs on the old code). 74 tests pass on macOS and on Linux (docker build --target test .).
  • Docs: the README explains how a pass works and how each failure is handled, and has a "Running" section with the exact commands and a note that the helper runs as a single replica.

Verified end to end against the svc dev stack (svc main including #14), natively and from the Docker image:

  • All 11 clips got a current probe with the right resolution, duration and frame rate.
  • A second pass found nothing t
  • A re-uploaded clip with a newctly once.
  • A text file named .mp4 was ste and not listed again.
  • Stopping the server mid-run ghelper reconnected after 1 min.
  • docker stop exits 0 in under

Not covered: SIGTERM while ffprnly tested between passes), andan HDR file over the real serve

Finishes the helper: it now runs until stopped, probes each file version once and reports the result to the server. - HelperRunner: - On start and then every POLL_INTERVAL_SECONDS, it fetches GET /media/unprobed until nothing new is listed. For each file it gets a download URL, runs ffprobe and PUTs the result. - Transient failures back off per media id and etag (1 min doubling to 1 h, held in memory). If a PUT fails transiently, the probe result is kept and resent without running ffprobe again. - 409 and 404 drop the file; 400/422 are logged at error and backed off. - If the server is unreachable, it logs one warning, backs off the same way and never exits. - main wires up the config, client, prober and runner. SIGTERM/SIGINT kill a running ffprobe and exit 0; only a bad configuration exits non-zero. - Logging: one Probed line per file (key, container, duration, resolution, hdr), a warning with the attempt count for each transient failure, and a Pass finished summary per pass (logged at debug when the pass did nothing). - Prober fixes: - hdr is now true/false for every probed video stream, as the contract says. - It waits for ffprobe with a termination handler instead of Process.waitUntilExit(), which sometimes never returned and hung the helper for good. - It closes ffprobe's pipes; each run used to leak two file descriptors. - Tests: the runner runs against a fake server on the stub, with a scripted prober and a fake clock. There's also a regression test that runs ffprobe 200 times in a row (it hangs on the old code). 74 tests pass on macOS and on Linux (docker build --target test .). - Docs: the README explains how a pass works and how each failure is handled, and has a "Running" section with the exact commands and a note that the helper runs as a single replica. Verified end to end against the svc dev stack (svc main including #14), natively and from the Docker image: - All 11 clips got a current probe with the right resolution, duration and frame rate. - A second pass found nothing t - A re-uploaded clip with a newctly once. - A text file named .mp4 was ste and not listed again. - Stopping the server mid-run ghelper reconnected after 1 min. - docker stop exits 0 in under Not covered: SIGTERM while ffprnly tested between passes), andan HDR file over the real serve
The contract says hdr is true for smpte2084 or arib-std-b67, else false.
An untagged or unknown transfer is SDR as far as the server is concerned,
so a probed video stream now always yields true or false.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
HelperRunner fetches GET /media/unprobed until it has nothing new, and for
each file gets a download URL, probes it and submits the result. Transient
failures wait out a per-file backoff (1 min doubling to 1 h, keyed by media
id and etag, in memory); a probe whose PUT failed transiently is resent
without running ffprobe again. 409 and 404 drop the file. An unreachable
server is logged once at warning and retried with the same backoff; run()
only returns when its task is cancelled.

Tested against a stateful fake of the helper API on the stub server, with
a fake prober and a manual clock.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The signal cancels the runner's task, which kills a running ffprobe and
ends the sleep between passes; the process then exits 0. Only a bad
configuration exits non-zero.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Process.waitUntilExit() sometimes never returned for an ffprobe that had
exited and been reaped: against the dev stack one probe in eleven hung the
runner for good, and since the wait blocked a thread, SIGTERM could not
stop it either. The termination handler is set before run() and awaited
without holding a thread. Both read ends are now closed once drained;
each run used to leak two descriptors. A kill (timeout or cancellation) is
skipped once the process has exited, so it cannot hit a reused pid.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README: what a pass does, how each failure is handled, the log lines, and a
Running section with the exact commands against the svc dev stack, natively
and from the image, and that the helper runs as a single replica.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
sophie merged commit 6ce5941cc3 into main 2026-10-16 22:31:03 +11:00
sophie deleted branch run-loop 2026-10-16 22:32:50 +11:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
sophie/simpleplayer-helper!5
No description provided.