AVX2 was only the first illegal instruction

I thought I had an AVX2 problem. I did, but fixing only AVX2 left another illegal instruction waiting behind it.

The hackathon judge exposed a CPU profile with AVX but no AVX2. The prebuilt llama.cpp wheel expected AVX2, so loading it on that machine could terminate the process in native code before Python produced a traceback.

I had already written a guard for that case and then forgotten about it:

def _cpu_supports_wheel():
    try:
        with open("/proc/cpuinfo") as f:
            return "avx2" in f.read()
    except OSError:
        return True

The guard refused to load the wheel when avx2 was missing. That prevented the crash, but it also let the rest of the pipeline return fallback answers. For days I read those scores as evidence that the local model was bad. The model had not run.

The first portable build was not the answer

The first fix attempt disabled AVX2, FMA, and F16C and added a bypass for the old guard. I put the bypass in the wrong function, one level below the check that still controlled loading. The image went through the same fallback path.

After moving the bypass and adding tests around it, the next image failed again. This time the wheel itself still contained an instruction the target CPU did not support.

The clue was in ggml's CMake configuration:

option(GGML_BMI2 "ggml: enable BMI2" ${INS_ENB})

For this non-native build, BMI2 was still enabled even though AVX2, FMA, and F16C were off. The target also lacked BMI2. When the binary reached pdep, a BMI2 instruction, the process died for the same basic reason an AVX2 instruction would have: the CPU could not execute the bytes it had been given.

Turning BMI2 off produced a wheel that survived on the judge. Its score was still bad, but the container lived. That separated instruction compatibility from the performance problem that came next.

The guard checked the wheel, not portability

The old guard answered one narrow question: can this CPU load the prebuilt AVX2 wheel? It did not prove that a replacement wheel was compatible, because it checked only one feature flag.

That is the mistake I want to keep from this. CPU compatibility is a contract between one binary and one target feature set. AVX2 and BMI2 are separate parts of that contract. This target lacked both; another target needs its own list.

Calling a build portable does not make it so. The build flags have to match the target, and the result still needs to run under that target's instruction profile. In this case, disabling the famous flag fixed only the first half of the problem.

---

This is one thread from five days debugging a black-box hackathon judge that returned one accuracy number and no logs. The full write-up is "One number, no logs".