Do You Need a GPU for Jellyfin? Usually Not, Here's What Actually Matters

Whenever a remote Jellyfin server starts buffering, the first instinct is "I need a beefier box" or "I need a GPU." Most of the time, you don't. The real problem is the server transcoding when it shouldn't be, and that is a config problem, not a hardware one.

We run media servers for a living, so we see this every day. Here is how to tell what is happening, and how to keep your setup off the transcode path so modest hardware is plenty.

The one decision that drives everything

Jellyfin can deliver a file four ways, cheapest to most expensive for your server:

Only the last one re-encodes video, and that is the only part that needs real horsepower. Stay in Direct Play and even a small CPU only box is fine. The whole game is not kicking off a video transcode by accident.

What actually forces a transcode

Three things, and you control all of them:

• The client can't handle the video codec or container.
• The stream is over the client or internet bitrate cap.
• An image based subtitle has to be burned in.

Fix those and you have solved most remote buffering without spending a cent on hardware.

Keep playback in Direct Play

• Match your library to what your clients can actually play. H.264 is the safe universal one, most modern clients also handle HEVC, and MP4 or MKV containers are widely supported. If everything direct plays, the server is barely working.
• Keep text subtitles around. Text subs (SRT, ASS) get handed to the client or remuxed, which is cheap. Image subs (PGS, VOBSUB) often have to be burned into the picture, and burn in is the heaviest transcode there is, because it is two encodes happening at once. Bazarr can fetch or OCR subs to text so a subtitle toggle doesn't suddenly hammer your server.
• Don't go 4K only for remote. A 4K HDR file to a remote client that can't direct play it becomes a heavy transcode plus an HDR to SDR tone map, because Jellyfin doesn't pass HDR through. That is the single most expensive thing the server does. Keep a 1080p copy for remote, and set the Internet streaming bitrate limit (Dashboard > Playback, and per user under Dashboard > Users) below your upload speed. A good rule of thumb is around 20 Mbps of upload before remote feels comfortable, and leave yourself headroom (we keep roughly 20 to 30 percent of upload free for everything else). One caveat: the cap is best effort, and a few clients (Infuse, some Android players) have open bugs where they ignore it, so keep that 1080p copy regardless.

Do this and a plain CPU only box handles a household fine, because it is hardly ever transcoding.

If you genuinely do need to transcode

Then yes, you want hardware acceleration, with two honest caveats.

First, most remote, VPS, and shared boxes don't even have a usable GPU. Intel chips ending in F have no integrated graphics, and plenty of Xeon and EPYC servers ship without an iGPU. So for a lot of people the real answer is to lean on Direct Play, not to go GPU shopping.

If you do have a GPU:

• Make sure it actually made it into the container, not just toggled on in the dashboard. NVIDIA: docker exec -it jellyfin nvidia-smi. Intel or AMD on Linux: sudo /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128. If either errors, you are quietly running on the CPU.
• Use jellyfin-ffmpeg (the build whose version string ends in -Jellyfin). A generic ffmpeg only gives partial acceleration.
• Consumer GeForce cards cap at 8 concurrent encode sessions per machine, not per card, so a second card doesn't raise it. Intel QSV and AMD VAAPI have no session limit. Worth knowing before you count on one card for a lot of streams.

And if you do transcode, put the transcoding temp path on local SSD or NVMe, not the same NFS or SMB mount as your library, or you will stutter while the GPU sits idle.

If your library is on a network mount

Real time monitoring uses inotify, and NFS and rclone mounts do not fire inotify events, so newly added files never appear until a scan runs. Use a scheduled library scan, or have your import tooling call the Jellyfin scan API when an import finishes.

Quick checklist

• Check whether you are even transcoding (Dashboard > Playback shows the method per active stream).
• Keep your library in Direct-Play-friendly codecs and containers.
• Use text subtitles, not image subtitles, where you can.
• Keep a 1080p copy for remote and set a sensible internet bitrate cap.
• Only chase a GPU once you have confirmed you genuinely transcode a lot.

FAQ

Do you need a GPU for Jellyfin?

Usually not. If your clients can Direct Play your files, the server barely does any work and a normal CPU is plenty. You only need a GPU when you regularly transcode, especially 4K or HDR, because software transcoding of those can't keep up in real time.

Why does my Jellyfin server only buffer when I watch remotely?

Remote clients hit your upload bandwidth and are more likely to trigger a transcode (a lower bitrate cap, an unsupported codec, or subtitles). Local clients on gigabit usually Direct Play. If remote buffers, check whether the session is transcoding and whether you have set a sensible remote bitrate limit.

How do I stop Jellyfin from transcoding?

Keep your media in formats your clients support, use text subtitles instead of image based ones, keep a 1080p copy for remote viewing, and set the internet streaming bitrate limit below your upload. That keeps playback in Direct Play, which is the cheap path.

Want this handled for you?

Here is the nice part: most of this is config, not hardware, which means you can get great Jellyfin performance on modest, managed hosting. That is our whole thing at Bytesized Hosting. Every plan runs the same full Jellyfin and arr stack, set up with sane Direct-Play-friendly defaults, and there is a real support team for when something acts up. No hardware for you to babysit.

If you would rather run it yourself, everything above is what we'd set up anyway.