RTX 3090 BF16 at Full Rate: What It Means for AI

RTX 3090 BF16 tensor math runs at half the rate of the professional cards built on the same chip, and so do FP16 with FP32 accumulation and TF32. A free Linux kernel module released on October 17, Zagitta's gpu-unlock, lifts that limit on GeForce RTX 30 series cards. With it loaded at boot, users report dense BF16 matrix throughput up 25 to 80 percent on an RTX 3090, 3080, 3070 or 3060. For local AI that means faster prompt processing (prefill) and image generation, but no faster token generation, because decoding is limited by memory bandwidth.

This guide explains what the module changes, the measured gains people have posted, which cards work, what it means for vLLM, llama.cpp and diffusion, and the risks you accept by running it.

Why RTX 3090 BF16 runs at half rate

Every GA102 class chip has the same tensor cores, but NVIDIA ships them with different speed settings. Its own Ampere GA102 architecture whitepaper lists the numbers side by side:

Look at the RTX 3090 row. FP16 with FP16 accumulation runs at 142 TFLOPS, but the moment you ask for FP32 accumulation the spec drops to 71, and BF16 is only listed with FP32 accumulation, so BF16 work always gets the slower number. The professional RTX A6000 has no such gap. INT8 has no gap either, which matters later.

The project README describes the mechanism. The module writes the chip's speed selector registers to the full BF16 rate and then resets the card, all before the normal NVIDIA driver binds to it. It reaches those registers through a flaw in NVIDIA's signed boot firmware, documented in the repo's own technical write up: a buffer overflow in the Booter firmware lets a payload perform a single memory write. The driver itself stays unmodified.

The change is not permanent. A power off resets the selectors, which is why the module runs at every boot and again after suspend. Remove the kernel option and the card is back to stock on the next cold boot.

Measured gains from user reports

The README's headline test is an RTX 3090 Ti running a dense 8192 cubed matrix multiply at 450 W with FP32 accumulation (median of five runs):

Other owners posted their own numbers in the issue tracker, and the README collects them:

• RTX 3090: BF16 GEMM gain of 25 to 67 percent across seven reports. The low end is from cards capped at 225 W.
• RTX 3090 Ti: 68 to 80 percent.
• RTX 3080 10 GB: 70 to 73 percent. The memory modded 3080 20 GB showed 56 to 59 percent.
• RTX 3070: 73 to 81 percent.
• RTX 3060 12 GB: about 80 percent.

These are matrix benchmarks, not model benchmarks, and each tester used their own script, so treat them as a range rather than a promise. Power limit has a big effect: a card held to 225 W cannot run the extra math as fast as one allowed 350 W or more.

What it means for local LLMs

Running an LLM has two phases, and the mod only helps one of them.

Prefill is when the model reads your prompt. All prompt tokens go through each layer together as large matrix multiplies, so it is limited by compute. This is where the mod shows up.

Decode is when the model writes its answer one token at a time. Each new token needs the full set of weights read from VRAM, so the speed limit is memory bandwidth (936 GB/s on an RTX 3090 per NVIDIA's whitepaper). Doubling the math rate does nothing for that. If you want to understand why bandwidth rules decode, the VRAM guide covers the weight and KV cache math.

The posted LLM tests match that split:

The README summarises all four LLM reports as a prefill gain of 14 to 36 percent, with decode and INT8 work about the same and outputs bit identical before and after in the reports that compared them.

So who notices? You will feel it with long prompts: retrieval over big documents, coding tools that send whole files, agents that resend long histories, and batch jobs that score many prompts. You will barely notice it in a short chat where the reply is long and the prompt is short.

Notice also that the GGUF engine gained less than vLLM running BF16 weights. A likely reason: many quantized kernels lean on integer math, and INT8 already ran at full rate on GeForce, so there was less to gain. Our GGUF quantization guide and the SGLang vs vLLM vs llama.cpp comparison explain how those engines differ.

Image generation and fine tuning

The README lists diffusion and FlashAttention among the matrix heavy work that speeds up, and the RTX 3090 report in issue 25 ran image and video generation alongside LLM serving for eight hours without errors. No per model image timing has been posted yet, so we won't quote one. If you run FLUX or other open image models in ComfyUI, you can measure your own seconds per image before and after; our FLUX open weights guide lists which models fit a 24 GB card.

Fine tuning with BF16 mixed precision in PyTorch runs its matrix multiplies through exactly the path that was halved, so it is a natural candidate. Nobody in the tracker has posted a training A/B yet, though, and training also spends time on memory traffic and optimizer steps that the mod does not touch. Test one epoch of your own LoRA or QLoRA job with Unsloth on both settings before you assume a number.

Which cards work

The module accepts GA102, GA103, GA104 and GA106 GeForce cards. Reported results so far:

RTX 40 and 50 series cards are not Ampere chips and are not covered by this project. Reports include up to four cards in one machine, the card that drives the display, and cards passed through to a virtual machine.

How setup works (Linux only)

The project ships as a Linux kernel module with packages for Arch, Debian and Ubuntu, and Fedora. In outline:

1. Clone the repo and run make install, which detects your distribution, builds the package and installs the module plus firmware into the initramfs.
2. Add gpu_unlock.targets=all (or a specific PCI address) to your kernel command line through GRUB, systemd-boot or your UKI setup.
3. Reboot from a full power off and check journalctl -k -b | grep gpu_unlock for the success line naming your card.

git clone https://github.com/Zagitta/gpu-unlock ~/bf16-mod
cd ~/bf16-mod
make install
# then add gpu_unlock.targets=all to the kernel command line and reboot
journalctl -k -b | grep gpu_unlock

Two traps showed up in the tracker. On Ubuntu systems where the NVIDIA driver sits inside the initramfs, the driver can bind the card first and the module fails with error 16 (issue 11). And every success report we read ran with Secure Boot turned off, because the module is unsigned and out of tree, so the kernel logs itself as tainted. Loading the module by hand after boot does nothing to a card the NVIDIA driver already owns.

Windows users should note this project has no Windows build. One report mentions a separate UEFI port working on Windows for a 3070 Ti, but that is outside this project.

Pros and cons

Pros

• Large gains in compute bound work: BF16 GEMM up 25 to 80 percent and LLM prefill up 14 to 36 percent in posted reports.
• Free, open source module (GPL 2.0) that leaves the NVIDIA driver unmodified, so driver updates do not need repatching.
• Reversible: remove the kernel option and cold boot to return to stock.
• Reported outputs are bit identical, so model quality does not change.

Cons

• Relies on exploiting a firmware flaw. It is a community project, not an NVIDIA feature, and a future driver or firmware change could stop it working.
• No change to decode speed, which is what most chat users feel.
• Linux only, Secure Boot off in practice, and a boot order trap on some Ubuntu setups.
• Gains depend on power limit, so cards held to low wattage see the smallest uplift.
• The firmware notice says redistribution approval is not claimed; it is meant for personal local packaging.

Who it's for

It suits people who already run RTX 30 series cards on Linux for long context serving with vLLM or SGLang, batch inference, diffusion, or BF16 fine tuning, and who are comfortable with kernel modules and boot options. It is a poor fit if your work is mostly short chats in Ollama or LM Studio, if you run Windows, or if the machine is a shared work box where an exploit based module would worry your administrator. If you are choosing hardware from scratch, memory size and bandwidth still decide which models you can run at all; see how much VRAM you need and running models with speculative decoding, which attacks the decode bottleneck instead.

FAQ

Does the BF16 mod work on the RTX 4090 or 40 series?

No. The project only targets Ampere GeForce chips (GA102, GA103, GA104 and GA106), which means RTX 30 series cards. RTX 40 series cards use a different chip generation and are not listed.

Does the mod make LLMs generate tokens faster?

Not in any measurable way. Every posted report shows decode speed flat, within run to run noise, because token generation is limited by memory bandwidth. What gets faster is prompt processing, by 14 to 36 percent in the reports, which shortens the wait before the first token on long prompts.

Is the BF16 mod safe for my RTX 3090?

Owners have reported clean boots, no driver errors and, in one RTX 3090 test, eight hours of mixed serving and image generation with temperatures and power at the 350 W cap unchanged from stock. That is user evidence, not a guarantee. It uses a firmware exploit, so treat it as an experiment and keep the kernel option easy to remove.

Does RTX 3090 support BF16?

Yes. NVIDIA's whitepaper lists BF16 tensor throughput for the RTX 3090 at 71 TFLOPS dense with FP32 accumulation, half of the RTX A6000's 154.8 on the same GA102 chip. The kernel module is about restoring the full rate, not adding BF16 support.

Where is the project on GitHub?

It is the repository published by the developer Zagitta, linked at the top of this guide. Its README is at version 1.0.0, and the repository went public on October 17, 2026. The issue tracker holds the compatibility reports and benchmarks quoted in this guide.

Will it help llama.cpp and GGUF models?

Less than it helps vLLM with BF16 weights. One report saw under 1 percent prefill gain at 8K context in a GGUF engine, rising to 12 to 13 percent at 32K and 128K. Integer math already ran at full rate on GeForce cards, so quantized paths have less to gain.