Skip to content

Add ARM64 crypto as NEON C intrinsics for MSVC (WOLFSSL_ARMASM_INTRINSICS) - #11035

Open
coneco-cy wants to merge 2 commits into
wolfSSL:masterfrom
coneco-cy:arm64-msvc-neon-intrinsics
Open

Add ARM64 crypto as NEON C intrinsics for MSVC (WOLFSSL_ARMASM_INTRINSICS)#11035
coneco-cy wants to merge 2 commits into
wolfSSL:masterfrom
coneco-cy:arm64-msvc-neon-intrinsics

Conversation

@coneco-cy

@coneco-cy coneco-cy commented Aug 3, 2026

Copy link
Copy Markdown

Description

wolfSSL's ARM64 crypto kernels ship in two forms today:

  • armv8-*-asm.S / armv8-*-asm_c.c — GNU/GAS assembly and GCC inline asm
  • armv8-*-asm.asm — Microsoft syntax, assembled by armasm64.exe

The armasm64 form already covers MSVC on Windows ARM64 and is wired up in
wolfssl.vcxproj (WolfSSLAarch64Asm=true). This PR does not replace it.

What it adds is a third form of the same kernels — portable NEON C
intrinsics
— because the armasm64 route requires a separate assembler
invocation, and in practice that is only wired up through MSBuild
CustomBuild rules. Projects that consume wolfSSL through CMake, or any
build system without an ARM64 assembler step, cannot use it as-is and silently
fall back to portable C for AES, AES-GCM (including the table-based GHASH),
SHA-256, Poly1305 and ChaCha20.

Intrinsics are ordinary C, so cl.exe compiles them directly: no assembler
step, no generated .asm, and no per-source target-feature flag (MSVC enables
NEON crypto unconditionally on ARM64, unlike clang which needs
-march=armv8-a+crypto).

This came out of profiling MariaDB — which bundles wolfSSL and builds it with
CMake — on a native Windows ARM64 machine, where GHASH and AES showed up as the
top crypto hot spots purely because the hardware path was unreachable from that
build system.

What is added

Four new sibling files under wolfcrypt/src/port/arm/, plus 44 lines of build
wiring. No existing crypto source is modified.

File Symbols
armv8-aes-intrinsics-msvc.c 19 AES / AES-CBC / AES-CTR / AES-ECB / AES-GCM (one-shot and streaming) *_AARCH64 kernels, plus the 7 base-layer symbols aes.c links unconditionally
armv8-sha-intrinsics-msvc.c Transform_Sha256_Len_crypto
armv8-poly1305-intrinsics-msvc.c poly1305_set_key, poly1305_arm64_block_16, poly1305_arm64_blocks, poly1305_final
armv8-chacha-intrinsics-msvc.c wc_chacha_setkey, wc_chacha_setiv, wc_chacha_use_over, wc_chacha_crypt_bytes — 4-block-parallel

These supply the same *_AARCH64 symbols as the assembly, so they are an
alternative to it, not an addition — hence an explicit opt-in rather than an
automatic fallback. Two independent layers keep existing builds untouched:

  1. A new WOLFSSL_ARMASM_INTRINSICS option, default off, which FATAL_ERRORs
    if requested on a non-MSVC toolchain.
  2. Each file self-guards on
    WOLFSSL_ARMASM && WOLFSSL_ARMASM_INTRINSICS && _MSC_VER && !__clang__ && (_M_ARM64 || _M_ARM64EC),
    so it compiles to nothing otherwise. (!__clang__ matters: clang-cl defines
    _MSC_VER but ships clang's arm_neon.h, where vmull_p64's low form takes a
    scalar poly64_t rather than MSVC's __n64.)

Why no changes to existing sources were needed

Worth noting explicitly, since an earlier version of this work did need them —
current master already solves both prerequisites:

  • settings.h:388
    maps _M_ARM64__aarch64__ under WOLFSSL_ARMASM, so the ~104
    __aarch64__ gates in aes.c select correctly under MSVC.
  • cpuid.c:473
    has a _WIN32 branch using IsProcessorFeaturePresent(), so
    Check_CPU_support_HwCrypto() works — no GCC-only mrs probe needed.

Because HAVE_CPUID_AARCH64 derives from __aarch64__ && WOLFSSL_ARMASM, that
bridge also enables the Windows cpuid path automatically.

Testing

All on a native Windows ARM64 machine (Snapdragon X2 Elite / Oryon),
MSVC 14.44.35207, SDK 10.0.26100.0.

Each kernel was validated against three independent references, not one:

  1. Byte-exact vs the original inline asm. The upstream armv8-*-asm_c.c
    bodies were extracted verbatim by script (never retyped) into a reference TU
    built with clang-cl (cl.exe cannot compile GCC inline asm), then compared
    over randomized inputs and edge cases — AES 100k random blocks × all three key
    sizes; CBC over block counts 1/2/3/7/16/255; ChaCha over a ragged multi-call
    sequence (5,1,10,64,3,100,63,65,200,7 bytes), which is the only shape that
    exercises the over/left leftover-keystream contract.
  2. Independent pure-C oracles written from the specs (AES-ECB, GF(2^128)
    GHASH, FIPS 180-4 SHA-256/512, RFC 8439 Poly1305/ChaCha20).
  3. Published KATs: FIPS-197 AES-128 and key schedule, SP800-38A AES-128-CBC
    (F.2.1) and AES-128-CTR (F.5.1, including a 20-byte partial tail), NIST GCM
    case 4 (ciphertext + tag + valid-decrypt + corrupt-tag reject), FIPS 180-4
    SHA-256/512, RFC 8439 §2.4.2 and §2.5.2.

Performance vs. portable C

Case portable C intrinsics speedup
AES-256-CBC encrypt 245 MB/s 1173 MB/s 4.8x
AES-256-CTR 266 MB/s 3234 MB/s 12.2x
AES-256-ECB encrypt 296 MB/s 3076 MB/s 10.4x
AES-256-GCM encrypt 74 MB/s 3087 MB/s 41.5x
AES-256-GCM decrypt 77 MB/s 2986 MB/s 38.9x

In-process profiles, symbol swap visible in-trace:

Kernel portable C intrinsics speedup
SHA-256 1166.3 ms 208.0 ms 5.6x
ChaCha20 244.8 ms 125.0 ms 2.0x
Poly1305 212.7 ms 87.0 ms 2.4x

Intrinsics vs. the armasm64 assembly

Since the point of this PR is an alternative to the assembly, I also compared the
performance difference between intrinsics and armasm64 assembly. Both
kernels were linked into the same binary from the same benchmark source, so only
the implementation differs.

The figures below use an idle machine, a 20 s
cooldown between every single run, discarded warm-up runs, and interleaved
A/B/A/B ordering. Checksums were verified identical between the two builds, so
both sides provably did the same work. 256 MiB per case, AES-256.

Case intrinsics (mean [range]) armasm64 (mean [range]) asm/intrin
AES-256-ECB enc 5571 [5376–5658] 5638 [5480–5750] 1.012 (ranges overlap)
AES-256-CTR 6875 [6294–7136] 7057 [6610–7269] 1.026 (ranges overlap)
AES-256-CBC enc 1179 [1149–1210] 1090 [1084–1096] 0.924
GHASH block 1905 [1849–1943] 1964 [1955–1982] 1.031

ECB and CTR are indistinguishable at this precision; CBC is ~8% faster in
intrinsics and GHASH ~3% slower. So the intrinsics are performance-equivalent
to the hand-written assembly
— no case is at a material disadvantage.

Checklist

  • added tests — external GoogleTest + KAT harnesses with proven negative
    controls; not yet integrated into wolfSSL's own suite, see question 5
  • updated/added doxygen
  • updated appropriate READMEs
  • Updated manual and documentation

…SICS)

wolfSSL's ARM64 crypto kernels ship in two forms: GNU/GAS assembly with GCC
inline asm (armv8-*-asm.S / -asm_c.c), and Microsoft-syntax assembly assembled
by armasm64.exe (armv8-*-asm.asm). The armasm64 route covers MSVC ARM64 and is
wired up in wolfssl.vcxproj via WolfSSLAarch64Asm.

That route needs a separate assembler invocation, which in practice is only
wired up through MSBuild CustomBuild rules. Projects consuming wolfSSL through
CMake - or any build without an ARM64 assembler step - cannot use it as-is, and
fall back to portable C for AES, AES-GCM (including table-based GHASH),
SHA-256, Poly1305 and ChaCha20.

This adds a third form: the same kernels as portable NEON C intrinsics, which
cl.exe compiles as ordinary C. No assembler step, no generated .asm, and no
per-source target-feature flag (MSVC enables NEON crypto unconditionally on
ARM64, unlike clang which needs -march=armv8-a+crypto).

The four new files supply the same *_AARCH64 symbols as the assembly, so they
are an alternative to it rather than an addition. They are opt-in behind
WOLFSSL_ARMASM_INTRINSICS and each is self-guarding on
_MSC_VER && !__clang__ && _M_ARM64, so every existing configuration and
toolchain is unaffected.

  armv8-aes-intrinsics-msvc.c       19 AES/AES-GCM/AES-CBC *_AARCH64 kernels
                                    plus 7 base-layer fallback symbols
  armv8-sha-intrinsics-msvc.c       Transform_Sha256_Len_crypto
  armv8-poly1305-intrinsics-msvc.c  4 Poly1305 kernels
  armv8-chacha-intrinsics-msvc.c    4 ChaCha20 kernels, 4-block-parallel

No existing crypto source is modified: settings.h already bridges
_M_ARM64 to __aarch64__ under WOLFSSL_ARMASM, and cpuid.c already has a
Windows ARM64 IsProcessorFeaturePresent() path, so the gates and the runtime
feature probe work as-is.

Each kernel was verified against three independent references: the original
inline asm extracted verbatim and built with clang-cl, independent pure-C
oracles, and published KATs (FIPS-197, FIPS 180-4, SP800-38A CBC/CTR, NIST
GCM, RFC 8439). Every test's failure path was proven first with an injected
negative control.

Not yet ported: AES_XTS_* (WOLFSSL_AES_XTS) and the _EOR3 variants
(WOLFSSL_ARMASM_CRYPTO_SHA3). A verified SHA-512 kernel is included but left
inert pending a wired-up feature gate.
@wolfSSL-Bot

Copy link
Copy Markdown

Can one of the admins verify this patch?

@dgarske

dgarske commented Aug 3, 2026

Copy link
Copy Markdown
Member

Hi @coneco-cy , thank you for submitting this work. I've asked @SparkiDev to review it and see if this is something we'd consider accepting. Can you tell us more about your project and interest in wolfSSL? If we are to accept this we will need to get a signed contributor agreement in place. Thanks, David Garske, wolfSSL

The first version of these kernels was correct but measurably slower than the
armasm64 path. Profiling the generated code on a Snapdragon X2 Elite (Oryon)
found two distinct causes, each isolated with a controlled experiment:

1. The rounds were driven by a runtime `nr`, so MSVC emitted them as a real
   loop - 3 scalar bookkeeping ops (sub/add/cbnz) per 2 AES instructions. At a
   fixed 1 block in flight, unrolling alone measured 2732 -> 5913 MB/s (2.16x).
   Fixed by dispatching on nr to three compile-time-constant bodies; AES has
   only three key sizes, so three instantiations cover every case.

2. ECB/CTR processed one block at a time, which is latency-bound: aese->aesmc
   is a serial chain with nothing to interleave. A pipeline-depth sweep (same
   instruction mix per block, only the number of independent chains varying)
   measured 2503 / 3540 / 4310 / 8024 MB/s at 1 / 2 / 4 / 8 blocks. Upstream's
   assembly avoids this with its _start_2/4/8 tiers; this now does 8 wide.

A third, non-obvious problem showed up while fixing (2): with the eight states
passed as `uint8x16_t s[8]`, MSVC keeps them in MEMORY and still emits a
single-block chain - the disassembly showed 42 load/stores for 63 AES ops and
the wide path was no faster. The lanes must be named locals for the compiler to
keep them in v-registers, hence the AES_X8_* macros. Similarly the single-block
helpers need __forceinline: the three unrolled bodies push them past MSVC's
inline heuristic, which otherwise emits a `bl` per block.

Measured after the change, on an idle machine with a 20s cooldown between every
run (sustained back-to-back runs thermally downclock this part from 4454 to
1382 MHz, which silently distorts any comparison), 4 interleaved pairs,
256 MiB per case, checksums verified identical between the two builds:

  case         intrinsics            armasm64              asm/intrin
  AES-256-ECB  5571 [5376-5658]      5638 [5480-5750]      1.012  (overlap)
  AES-256-CTR  6875 [6294-7136]      7057 [6610-7269]      1.026  (overlap)
  AES-256-CBC  1179 [1149-1210]      1090 [1084-1096]      0.924
  GHASH block  1905 [1849-1943]      1964 [1955-1982]      1.031

ECB was 1.97x slower before this change; it is now within run-to-run variance.
No case is now at a material disadvantage to the hand-written assembly.

Correctness was re-verified against the armasm64 kernels linked into the same
binary (all three key sizes, block counts 1..20 covering every remainder class
around the new 8-block boundary, 63/94/125-block sizes, CTR partial tails
exercising tmp/*left, ECB decrypt) plus the FIPS-197 C.1 and SP800-38A F.5.1
vectors, and GHASH separately over 1..40 folded blocks. Each check was first
proven able to fail by injecting a negative control (dropping round 12 from the
unrolled body: 79 failures; reversing the 8-block store order: 87 failures;
GHASH reduction constant 0x87->0x86: red).

Also moves two declarations in AES_GCM_ghash_block_AARCH64 to the top of the
block: clang-cl's -Wdeclaration-after-statement flagged them as pre-C99
incompatible.
@coneco-cy
coneco-cy marked this pull request as ready for review August 5, 2026 03:29
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

retest this please

@coneco-cy

coneco-cy commented Aug 5, 2026

Copy link
Copy Markdown
Author

Can you tell us more about your project and interest in wolfSSL?

Hi @dgarske ,

Happy to give the background, since it explains why this is shaped the way it is.

I work at Qualcomm, on Windows-on-ARM64 (Snapdragon) enablement. One of our ISVs deploys MariaDB, and we are working to get MariaDB building and running well on Windows ARM64 with the MSVC toolchain. MariaDB bundles wolfSSL as its TLS/crypto provider (extra/wolfssl), so wolfSSL's ARM64 story on MSVC becomes our story too — that is the whole of my interest here.

The concrete problem we hit:
MariaDB builds wolfSSL through CMake, and its extra/wolfssl/CMakeLists.txt only ever enables an assembler for Intel:

IF(MSVC_INTEL)
  PROJECT(wolfssl C ASM_MASM)
ELSEIF(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|amd64")
  PROJECT(wolfssl C ASM)
ELSE()
  PROJECT(wolfssl C)      # <-- MSVC ARM64 lands here: no assembler at all
ENDIF()

So on MSVC ARM64 there is no ASM language enabled and no armasm64 step, and all of AES, AES-GCM (including table-based GHASH), SHA-256, Poly1305 and ChaCha20 fall back to portable C. That was visible in profiling as crypto hot spots on an otherwise capable machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants