nixos/docs/superpowers/specs/2026-07-26-jellyfin-hw-transcoding.md

Intel Quick Sync hardware transcoding for Jellyfin on jupiter

Problem

Jellyfin streams stutter on jupiter whenever a client needs a transcode (unsupported codec/container, bitrate cap, or a client that can't direct-play). Transcoding currently runs entirely on CPU.

jupiter's Intel iGPU is already usable at the OS level:

• hardware.graphics.enable = true with intel-media-driver (the iHD VAAPI driver) is configured in machines/jupiter/hardware-configuration.nix:20-27.
• The commented-out i915.force_probe = "9a49" kernel param there corresponds to a Quick-Sync-capable Intel UHD iGPU, confirming the hardware supports it.

But modules/environments/jellyfin/default.nix never grants the jellyfin systemd service access to /dev/dri, so Jellyfin has no path to the GPU and silently falls back to software transcoding.

Goal

Give the Jellyfin service access to the iGPU's VAAPI render node, so Quick Sync can be enabled in Jellyfin's own dashboard and transcodes are offloaded from the CPU.

Non-goals

• Remote/external access or reverse-proxy tuning.
• General CPU/RAM headroom review of jupiter.
• A toggle option (my.profiles.jellyfin.hardwareAcceleration.enable) — jupiter only has the one iGPU, so this is hardcoded on rather than made configurable.

Design

NixOS change (declarative)

In modules/environments/jellyfin/default.nix, inside the existing config = lib.mkIf cfg.enable { ... } block, grant the systemd service supplementary access to the video and render groups (the groups that own /dev/dri/card* and /dev/dri/renderD*):

systemd.services.jellyfin.serviceConfig.SupplementaryGroups = [
  "video"
  "render"
];

This is additive to the existing systemd.services.jellyfin.after = [ "network-online.target" ]; block already in the file — both apply to the same service.

Also add libva-utils to environment.systemPackages (or scoped to this module) so vainfo is available on jupiter to verify the driver loads correctly.

Manual step (not declarative)

Jellyfin stores its transcoding/hardware-acceleration choice in its own internal encoding.xml, which the NixOS module does not expose as an option. After deploying the Nix change, one-time manual configuration in the Jellyfin dashboard is required:

1. Dashboard → Playback.
2. Hardware acceleration: Intel QuickSync (QSV).
3. VA-API device: /dev/dri/renderD128.
4. Enable hardware decoding for the codecs your library actually uses (H264 at minimum; HEVC/VP9 depending on iGPU generation).
5. If any HDR content exists in the library, enable tone-mapping — this is one of the more CPU-expensive operations Quick Sync can offload.

Verification

Build-time (from the Mac, no SSH needed):

nix eval '.#nixosConfigurations.jupiter.config.systemd.services.jellyfin.serviceConfig.SupplementaryGroups' \
  --extra-experimental-features 'nix-command flakes'

Expect [ "video" "render" ].

On jupiter after sudo nixos-rebuild switch --flake '.#jupiter':

systemctl status jellyfin
journalctl -u jellyfin -n 50 --no-pager
vainfo

vainfo should list the iHD driver and print supported VAEntrypoints (VLD decode / encode profiles for H264/HEVC).

Functional check: play a file on a client that forces transcoding (or force it manually via Jellyfin's playback quality setting), then in Jellyfin's dashboard Activity/Now Playing panel confirm the transcode reason and check that CPU usage on jupiter (htop) stays low during playback rather than pegging a core — Quick Sync offload should show up as low CPU, some GPU (intel_gpu_top) activity instead.

Open items

• Exact supported codec list depends on the iGPU generation (device ID 9a49) — confirm via vainfo output once run, and enable only the hardware decode paths it actually reports.

Post-deploy fix: two additional runtime packages required

After the initial deploy (Task 1's SupplementaryGroups grant) and enabling QSV in the dashboard, HEVC HDR playback hung indefinitely (Direct Play worked for some titles; titles that needed a real transcode+tonemap never produced output). Root-caused via journalctl -u jellyfin and the per-session ffmpeg transcode log (find / -xdev -iname '*ffmpeg-transcode*') — two separate runtimes were missing beyond intel-media-driver (which only provides VAAPI):

1. QSV session creation failed: Error creating a MFX session: -9 / Error initializing an MFX session: -3 on -init_hw_device qsv=qs@va. VAAPI and QSV are separate runtimes on Linux — QSV needs the oneVPL/MFX GPU implementation. Fix: added pkgs.vpl-gpu-rt ("oneAPI Video Processing Library Intel GPU implementation"; note onevpl-intel-gpu is the old, renamed attribute) to hardware.graphics.extraPackages in machines/jupiter/hardware-configuration.nix.

2. OpenCL device creation failed: Failed to get number of OpenCL platforms: -1001 (CL_PLATFORM_NOT_FOUND_KHR) on -init_hw_device opencl=ocl@va. The tonemap_opencl filter jellyfin uses for HDR→SDR tone-mapping needs a working OpenCL ICD, which nothing installed so far provides. Fix: added pkgs.intel-compute-runtime ("Intel Graphics Compute Runtime oneAPI Level Zero and OpenCL, supporting 12th Gen and newer" — matches jupiter's Tiger Lake/Xe iGPU) to the same extraPackages list.

Confirmed working end-to-end: HEVC HDR transcode with QSV encode + OpenCL tone-map runs at speed=2.68x realtime on jupiter's iGPU, and plays smoothly on Apple TV (JellyTV app).

Both packages live in machines/jupiter/hardware-configuration.nix (hardware.graphics.extraPackages), alongside intel-media-driver, rather than in the jellyfin module itself — they're iGPU runtime capabilities, not something specific to the jellyfin service.