Feat/jellyfin hw transcoding #3

Merged
finn.markwitz merged 6 commits from feat/jellyfin-hw-transcoding into main 2026-07-27 09:56:11 +02:00
4 changed files with 307 additions and 0 deletions
@@ -0,0 +1,154 @@
# Jellyfin Hardware Transcoding (jupiter) Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Give jupiter's Jellyfin service access to the Intel iGPU's VAAPI render node so Quick Sync hardware transcoding can be enabled, instead of every transcode falling back to CPU.
**Architecture:** One NixOS module change (`modules/environments/jellyfin/default.nix`) grants the `jellyfin` systemd service supplementary access to the `video`/`render` groups and installs `libva-utils` for verification. This is declarative and build-verifiable from the Mac. Enabling Quick Sync inside Jellyfin's own dashboard, and the on-machine verification, is a manual step run by the user on jupiter after deploy — the NixOS module has no option for it and this environment's convention is that the assistant never SSHes into jupiter directly (see `docs/superpowers/specs/2026-07-26-jellyfin-hw-transcoding.md`).
**Tech Stack:** NixOS (flake-parts), nixpkgs `services.jellyfin` module, VAAPI/`intel-media-driver`, `libva-utils`.
## Global Constraints
- No SSH from the assistant into jupiter — all on-machine commands are given to the user to run and paste back.
- Follow the existing profile pattern in `modules/environments/jellyfin/default.nix` (`config = lib.mkIf cfg.enable { ... }`); don't introduce a new toggle option — hardcode the hardware-acceleration wiring on, per the approved spec.
- Verify locally via `nix eval` / `nix build` before asking the user to deploy.
---
### Task 1: Grant Jellyfin access to the iGPU and verify the build
**Files:**
- Modify: `modules/environments/jellyfin/default.nix`
**Interfaces:**
- Produces: `systemd.services.jellyfin.serviceConfig.SupplementaryGroups = [ "video" "render" ];` — verified via `nix eval` in Step 2.
- [ ] **Step 1: Add the device-access config and `libva-utils` package**
Read the current file first (`modules/environments/jellyfin/default.nix`), then edit the `config = lib.mkIf cfg.enable { ... }` block so it reads:
```nix
config = lib.mkIf cfg.enable {
services.jellyfin = {
enable = true;
openFirewall = true;
};
environment.systemPackages = [ pkgs.libva-utils ];
my.homepage.services = [
{
group = "Media";
name = "Jellyfin";
description = "Media server";
href = "http://${hostName}:${toString port}";
icon = "jellyfin.png";
}
];
systemd.services.jellyfin = {
after = [ "network-online.target" ];
serviceConfig.SupplementaryGroups = [
"video"
"render"
];
};
};
```
Note the two existing `systemd.services.jellyfin` keys (`after`) and the new `serviceConfig.SupplementaryGroups` now live in the same attrset — don't create a second `systemd.services.jellyfin = { ... }` block, it would overwrite the first.
- [ ] **Step 2: Verify the rendered config with `nix eval`**
Run (from the repo root on the Mac):
```bash
nix eval '.#nixosConfigurations.jupiter.config.systemd.services.jellyfin.serviceConfig.SupplementaryGroups' \
--extra-experimental-features 'nix-command flakes'
```
Expected output: `[ "video" "render" ]`
- [ ] **Step 3: Verify the machine still builds**
Run:
```bash
nix build '.#nixosConfigurations.jupiter.config.system.build.toplevel' \
--extra-experimental-features 'nix-command flakes' --no-link
```
Expected: build succeeds with no errors (may take a while; watch for any evaluation error mentioning `jellyfin` or `libva-utils`).
- [ ] **Step 4: Format and commit**
```bash
nixfmt-rfc-style modules/environments/jellyfin/default.nix
git add modules/environments/jellyfin/default.nix
git commit -m "feat(jellyfin): grant iGPU access for Quick Sync hardware transcoding"
```
---
### Task 2: Deploy on jupiter and enable Quick Sync (user-executed)
**Files:** none (on-machine deploy + Jellyfin dashboard UI)
**Interfaces:**
- Consumes: the `SupplementaryGroups` change from Task 1, already merged into the flake.
These steps run **on jupiter**, by the user — paste the output back so we can confirm each one before moving to the next.
- [ ] **Step 1: Deploy**
```bash
sudo nixos-rebuild switch --flake '.#jupiter'
```
Expected: switch succeeds, no errors mentioning `jellyfin`.
- [ ] **Step 2: Confirm the service picked up the new groups**
```bash
systemctl show jellyfin -p SupplementaryGroups
systemctl status jellyfin --no-pager
```
Expected: `SupplementaryGroups=video render` (order may vary) and the service is `active (running)`.
- [ ] **Step 3: Confirm VAAPI driver loads**
```bash
vainfo
```
Expected: output starts with something like `vainfo: VA-API version: 1.x` and `Driver version: Intel iHD driver`, followed by a list of supported VAProfiles/VAEntrypoints (e.g. `VAProfileH264Main : VAEntrypointVLD`, `VAEntrypointEncSlice`).
If this instead prints a permissions or "no VA display" error, paste it back — that means the group grant isn't reaching the process and Task 1 needs a follow-up fix (e.g. the jellyfin service may be more sandboxed than expected, requiring an explicit `DeviceAllow=char-drm rw` in `serviceConfig` as well).
- [ ] **Step 4: Enable Quick Sync in the Jellyfin dashboard**
In the Jellyfin web UI:
1. **Dashboard → Playback**.
2. Hardware acceleration: **Intel QuickSync (QSV)**.
3. VA-API device: `/dev/dri/renderD128`.
4. Enable hardware decoding for the codecs your library uses (H264 at minimum).
5. If the library has HDR content, enable tone-mapping.
6. Save.
- [ ] **Step 5: Functional check**
Play a file that requires transcoding (or force a lower quality in the client's playback settings to trigger one), then:
```bash
journalctl -u jellyfin -n 50 --no-pager
```
Look for a line referencing `qsv` or `vaapi` in the transcode command. Separately, watch CPU usage (`htop`) during playback — it should stay low on the core doing the transcode, rather than pegging at 100%, since the iGPU is now doing the encode/decode work.
## Self-Review Notes
- Spec coverage: NixOS change (Task 1) ✓, manual dashboard step (Task 2 Step 4) ✓, verification via `vainfo`/build (Task 1 Step 2-3, Task 2 Step 3) ✓, functional check (Task 2 Step 5) ✓. Toggle option explicitly excluded per approved spec — not present, correctly.
- No placeholders — every step has literal commands/code.
- `SupplementaryGroups` key/value matches exactly between Task 1 (produced) and Task 2 (consumed/checked).
@@ -0,0 +1,145 @@
# Intel Quick Sync hardware transcoding for Jellyfin on jupiter
## Problem
Jellyfin streams stutter on jupiter whenever a client needs a transcode
(unsupported codec/container, bitrate cap, or a client that can't
direct-play). Transcoding currently runs entirely on CPU.
jupiter's Intel iGPU is already usable at the OS level:
- `hardware.graphics.enable = true` with `intel-media-driver` (the `iHD`
VAAPI driver) is configured in
`machines/jupiter/hardware-configuration.nix:20-27`.
- The commented-out `i915.force_probe = "9a49"` kernel param there
corresponds to a Quick-Sync-capable Intel UHD iGPU, confirming the
hardware supports it.
But `modules/environments/jellyfin/default.nix` never grants the
`jellyfin` systemd service access to `/dev/dri`, so Jellyfin has no path
to the GPU and silently falls back to software transcoding.
## Goal
Give the Jellyfin service access to the iGPU's VAAPI render node, so
Quick Sync can be enabled in Jellyfin's own dashboard and transcodes are
offloaded from the CPU.
## Non-goals
- Remote/external access or reverse-proxy tuning.
- General CPU/RAM headroom review of jupiter.
- A toggle option (`my.profiles.jellyfin.hardwareAcceleration.enable`) —
jupiter only has the one iGPU, so this is hardcoded on rather than
made configurable.
## Design
### NixOS change (declarative)
In `modules/environments/jellyfin/default.nix`, inside the existing
`config = lib.mkIf cfg.enable { ... }` block, grant the systemd service
supplementary access to the `video` and `render` groups (the groups that
own `/dev/dri/card*` and `/dev/dri/renderD*`):
```nix
systemd.services.jellyfin.serviceConfig.SupplementaryGroups = [
"video"
"render"
];
```
This is additive to the existing `systemd.services.jellyfin.after = [
"network-online.target" ];` block already in the file — both apply to
the same service.
Also add `libva-utils` to `environment.systemPackages` (or scoped to
this module) so `vainfo` is available on jupiter to verify the driver
loads correctly.
### Manual step (not declarative)
Jellyfin stores its transcoding/hardware-acceleration choice in its own
internal `encoding.xml`, which the NixOS module does not expose as an
option. After deploying the Nix change, one-time manual configuration in
the Jellyfin dashboard is required:
1. **Dashboard → Playback**.
2. Hardware acceleration: **Intel QuickSync (QSV)**.
3. VA-API device: `/dev/dri/renderD128`.
4. Enable hardware decoding for the codecs your library actually uses
(H264 at minimum; HEVC/VP9 depending on iGPU generation).
5. If any HDR content exists in the library, enable tone-mapping — this
is one of the more CPU-expensive operations Quick Sync can offload.
## Verification
Build-time (from the Mac, no SSH needed):
```
nix eval '.#nixosConfigurations.jupiter.config.systemd.services.jellyfin.serviceConfig.SupplementaryGroups' \
--extra-experimental-features 'nix-command flakes'
```
Expect `[ "video" "render" ]`.
On jupiter after `sudo nixos-rebuild switch --flake '.#jupiter'`:
```
systemctl status jellyfin
journalctl -u jellyfin -n 50 --no-pager
vainfo
```
`vainfo` should list the `iHD` driver and print supported VAEntrypoints
(VLD decode / encode profiles for H264/HEVC).
Functional check: play a file on a client that forces transcoding (or
force it manually via Jellyfin's playback quality setting), then in
Jellyfin's dashboard **Activity/Now Playing** panel confirm the
transcode reason and check that CPU usage on jupiter (`htop`) stays low
during playback rather than pegging a core — Quick Sync offload should
show up as low CPU, some GPU (`intel_gpu_top`) activity instead.
## Open items
- Exact supported codec list depends on the iGPU generation (device ID
`9a49`) — confirm via `vainfo` output once run, and enable only the
hardware decode paths it actually reports.
## Post-deploy fix: two additional runtime packages required
After the initial deploy (Task 1's `SupplementaryGroups` grant) and
enabling QSV in the dashboard, HEVC HDR playback hung indefinitely
(Direct Play worked for some titles; titles that needed a real
transcode+tonemap never produced output). Root-caused via
`journalctl -u jellyfin` and the per-session ffmpeg transcode log
(`find / -xdev -iname '*ffmpeg-transcode*'`) — two separate runtimes
were missing beyond `intel-media-driver` (which only provides VAAPI):
1. **QSV session creation failed:** `Error creating a MFX session: -9`
/ `Error initializing an MFX session: -3` on
`-init_hw_device qsv=qs@va`. VAAPI and QSV are separate runtimes on
Linux — QSV needs the oneVPL/MFX GPU implementation. Fix: added
`pkgs.vpl-gpu-rt` ("oneAPI Video Processing Library Intel GPU
implementation"; note `onevpl-intel-gpu` is the old, renamed
attribute) to `hardware.graphics.extraPackages` in
`machines/jupiter/hardware-configuration.nix`.
2. **OpenCL device creation failed:** `Failed to get number of OpenCL
platforms: -1001` (`CL_PLATFORM_NOT_FOUND_KHR`) on
`-init_hw_device opencl=ocl@va`. The `tonemap_opencl` filter jellyfin
uses for HDR→SDR tone-mapping needs a working OpenCL ICD, which
nothing installed so far provides. Fix: added
`pkgs.intel-compute-runtime` ("Intel Graphics Compute Runtime oneAPI
Level Zero and OpenCL, supporting 12th Gen and newer" — matches
jupiter's Tiger Lake/Xe iGPU) to the same `extraPackages` list.
Confirmed working end-to-end: HEVC HDR transcode with QSV encode +
OpenCL tone-map runs at `speed=2.68x` realtime on jupiter's iGPU, and
plays smoothly on Apple TV (JellyTV app).
Both packages live in `machines/jupiter/hardware-configuration.nix`
(`hardware.graphics.extraPackages`), alongside `intel-media-driver`,
rather than in the jellyfin module itself — they're iGPU runtime
capabilities, not something specific to the jellyfin service.
@@ -35,6 +35,8 @@
#vaapiIntel # LIBVA_DRIVER_NAME=i965 (older but works better for Firefox/Chromium) #vaapiIntel # LIBVA_DRIVER_NAME=i965 (older but works better for Firefox/Chromium)
libva-vdpau-driver libva-vdpau-driver
libvdpau-va-gl libvdpau-va-gl
vpl-gpu-rt # oneVPL/MFX runtime, required for QSV (h264_qsv/hevc_qsv) session creation
intel-compute-runtime # OpenCL runtime, required for tonemap_opencl (HDR tone-mapping)
]; ];
}; };
@@ -22,6 +22,8 @@ in
openFirewall = true; openFirewall = true;
}; };
environment.systemPackages = [ pkgs.libva-utils ];
my.homepage.services = [ my.homepage.services = [
{ {
group = "Media"; group = "Media";
@@ -34,6 +36,10 @@ in
systemd.services.jellyfin = { systemd.services.jellyfin = {
after = [ "network-online.target" ]; after = [ "network-online.target" ];
serviceConfig.SupplementaryGroups = [
"video"
"render"
];
}; };
}; };
} }