The RSA module is essentially a hardware-accelerated modular exponentiation engine. It is specially optimized for RSA applications, so its behavior can be incoherent when RSA's invariants are broken.
Observed edge cases
- if 2 divides mod, output == 0
|0||Start (1=Enable/Busy, 0=Idle)|
|4-7||Keyslot (Bit6-7 don't actually affect the keyslot)|
|8||Endianness (1=Little endian, 0=Big endian). Affects RSA_EXPFIFO, RSA_MOD, and RSA_TXT.|
|9||Word order (1=Normal order, 0=Reversed order). Affects RSA_MOD and RSA_TXT.|
|0||Key status (1=Key has been set, 0=Key has not been set yet)|
|1||Key write-protect, this bit is RW. (0 = no protection, 1 = protected)|
Before writing RSA_EXPFIFO/RSA_MOD, bit0 here should be cleared when bit31 is already clear. Otherwise, the ARM9 will hang when attempting to write to RSA_EXPFIFO.
This contains the RSA size for this slot, in words. Normally this is 0x40 for RSA-2048.
The 0x100-byte private or public exponent is written to this write-only FIFO.
The RSA key modulus for the selected keyslot can be written here. When writing the RSA modulus, the modulus must align with the end of the register area.
Writing to RSA_MOD does not change the exponent written with RSA_EXPFIFO. An attack based on the Pohlig-Hellman algorithm exists to "read" the contents of RSA_EXPFIFO as a result (see 3DS System Flaws).
The RSA signature can be written here, and the data read from here is the message. When writing the RSA signature, the signature must be prepended with zeroes until it is a multiple of 8 bytes, and the end of the signature must align with the end of the register area.
The PKCS message padding must be manually checked by software, as hardware will only do raw RSA operations.
|1||CXI access desc (following the exheader)|
|2-3||Initialized by the ARM9 bootrom, but not used by any of the FIRMs. It's unknown what the ARM9 bootrom uses these for, if anything.|