Difference between revisions of "Savegames"

From 3dbrew
Jump to navigation Jump to search
 
(70 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This page describes the format and encryption of savegames contained in gamecards, SD/NAND, and SD/NAND [[extdata]]. You can find savegames from various 3DS games on the [[Games]] page.
+
This page describes the format and encryption of savegames contained in gamecards, SD and NAND. You can find savegames from various 3DS games on the [[Games]] page.
  
 +
== Overview ==
 +
Savegames are stored in [[DISA and DIFF|DISA container format]]. Inside the DISA container, it forms a [[Inner FAT|FAT filesystem]]. '''Please refer to these pages for how to fully extract save files'''. This page only describes additional encryption wear leveling on top of the DISA container. These layers only apply to gamecard save games. SD savegames and NAND savegames are DISA containers in plaintext after decrypting the common SD/NAND encryption layer.
  
=== Savegame Encryption ===
+
== Gamecard savegame Encryption ==
  
On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.
+
Gamecard encryption is AES-CTR applied on top of DISA container, but below the wear leveling layer (if exists). The same key Y used for encryption is also used for DISA CMAC signing. Several versions of encryption scheme have been introduced over the time.
 
 
The reason this works is because the stream cipher used has a period of 512 bytes. That is to say, it will repeat the same keystream after 512 bytes. The way you encrypt with a stream cipher is you XOR your data with the keystream as it is produced. Unfortunately, if your streamcipher repeats and you are encrypting a known plain-text (in our case, zeros) you are basically giving away your valuable keystream.
 
 
 
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.
 
 
 
The gamecard savegame [[AES|keyslot]] keyY is unique per gamecard. This keyY is unique for every region of each game.
 
 
 
All gamecard and NAND/SD savegames are encrypted with AES-CTR. The base CTR for gamecard savegames is all-zero. The CTR used for gamecard savegames eventually repeats, while NAND/SD savegames don't use a repeating CTR. For the old gamecard CTR method, it repeated every 0x200-bytes. With the new method it repeats at least every 0x1000-bytes, but the exact period isn't known for certain. NAND/SD savegames use a separate CTR method from the gamecard savegames, see the [[extdata]] page regarding extdata encryption.
 
  
With system version [[2.2.0-4]] the system now uses a different gamecard CTR method, which fixed the above flaw. The CTR seems to repeat in the image but not every 0x200 bytes. The CTR may repeat every 0x1000 bytes. The system uses the new method for titles which have the [[NCSD|CCI]] NVer version set to the [[2.2.0-4]] or above, starting with "Super Mario 3D Land". Prior to that NVer version, the system uses the old 0x200-byte CTR period.
+
{| class="wikitable" border="1"
 
 
=== Wear leveling ===
 
 
 
The 3DS employs a wear leveling scheme on the savegame FLASH chips. This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.
 
 
 
First, there are 8 bytes whose purposes are currently unknown. Then comes the actual blockmap.
 
The blockmap structure is simple:
 
<pre>
 
struct header_entry {
 
        uint8_t phys_sec; // when bit7 is set, block has checksums, otherwise checksums are all zero
 
        uint8_t alloc_cnt;
 
        uint8_t chksums[8];
 
} __attribute__((__packed__));
 
</pre>
 
 
 
There's one entry per sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).
 
 
 
The 2 bytes that follow the blockmap are the CRC16 (with starting value 0xFFFF (like modbus)) of the first 8 bytes and the blockmap.
 
 
 
Then comes the journal.
 
The journal structure is as follows:
 
<pre>
 
struct sector_entry {
 
        uint8_t virt_sec;      // Mapped to sector
 
        uint8_t prev_virt_sec;  // Physical sector previously mapped to
 
        uint8_t phys_sec;      // Mapped from sector
 
        uint8_t prev_phys_sec;  // Virtual sector previously mapped to
 
        uint8_t phys_realloc_cnt;      // Amount of times physical sector has been remapped
 
        uint8_t virt_realloc_cnt;      // Amount of times virtual sector has been remapped
 
        uint8_t chksums[8];
 
} __attribute__((__packed__));
 
 
 
struct long_sector_entry{
 
        struct sector_entry sector;
 
        struct sector_entry dupe;
 
        uint32_t magic;
 
}__attribute__((__packed__));
 
</pre>
 
 
 
With magic being a constant 0x080d6ce0.
 
 
 
The checksums in the blockmap/journal entries work as follows:
 
* each byte is the checksum of an encrypted 0x200 bytes large block
 
* to calculate the checksum, a CRC16 of the block (with starting value 0xFFFF) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum
 
 
 
=== MAC header ===
 
 
 
{| class="wikitable"
 
|-
 
! Image offset
 
! Length
 
! Description
 
 
|-
 
|-
| 0x00
+
!  FW Introduced
| 0x10
+
!  Old3DS
| [[AES]]-CBC MAC over a 0x20-byte SHA256 hash
+
[[AES#Keyslot|AES Keyslots]] (Encryotion / CMAC)
 +
!  KeyY generation method
 +
!  Repeating CTR
 
|-
 
|-
| 0x10
+
| The initial version
| 0xF0
+
| style="background: #ccffbb" | Yes
| Zero padding
+
| 0x37 / 0x33
|}
+
| v1
 
+
| style="background: #ccffbb" | Yes
This MAC is used to verify the DISA/DIFF header. SHA256_Update() is used to calculate the hash with the blocks described below.
 
 
 
==== Savegame Types ====
 
{| class="wikitable"
 
 
|-
 
|-
! Type
+
| [[2.0.0-2]]
! Description
+
| style="background: #ccffbb" | Yes
 +
| 0x37 / 0x33
 +
| v2
 +
| style="background: #ccffbb" | Yes
 
|-
 
|-
| CTR-EXT0
+
| [[2.2.0-4]]
| SD/NAND [[Extdata]]
+
| style="background: #ccffbb" | Yes
 +
| 0x37 / 0x33
 +
| v2
 +
| style="background: #ffccbb" | No
 
|-
 
|-
| CTR-SYS0
+
| [[6.0.0-11]]
| [[System SaveData]]
+
| style="background: #ccffbb" | Yes
 +
| 0x37 / 0x33
 +
| v3
 +
| style="background: #ffccbb" | No
 
|-
 
|-
| CTR-NOR0
+
| [[9.6.0-24|9.6.0-X]]
| Gamecard Savegame
+
| style="background: #ffccbb" | No
 +
| 0x1A / 0x19
 +
| v2?
 +
| style="background: #ffccbb" | No
 
|}
 
|}
  
==== Extdata SHA256 Blocks ====
+
=== Repeating CTR Fail ===
{| class="wikitable"
+
On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.
|-
 
! Block Size
 
! Description
 
|-
 
| 0x8
 
| Savegame type
 
|-
 
| 0x8
 
| First word is the hex ID from image filename, second word is the hex ID of the sub-dir under the <ExtdataIDLow> directory (all-zero for Quota.dat)
 
|-
 
| 0x4
 
| 1 for Quota.dat, 0 otherwise
 
|-
 
| 0x8
 
| Same as the previous u64
 
|-
 
| 0x100
 
| DIFF header
 
|}
 
  
 +
The reason this works is because the stream cipher used has a period of 512 bytes. That is to say, it will repeat the same keystream after 512 bytes. The way you encrypt with a stream cipher is you XOR your data with the keystream as it is produced. Unfortunately, if your streamcipher repeats and you are encrypting a known plain-text (in our case, zeros) you are basically giving away your valuable keystream.
  
==== System SaveData SHA256 Blocks ====
+
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.
{| class="wikitable"
 
|-
 
! Block Size
 
! Description
 
|-
 
| 0x8
 
| Savegame type
 
|-
 
| 0x8
 
| [[FS:OpenFile|SaveID]]
 
|-
 
| 0x100
 
| DISA header
 
|}
 
 
 
==== Gamecard Savegame SHA256 Blocks ====
 
{| class="wikitable"
 
|-
 
! Block Size
 
! Description
 
|-
 
| 0x8
 
| Savegame type
 
|-
 
| 0x100
 
| DISA header
 
|}
 
  
=== Partitions ===
 
  
There can be multiple partitions in the image.
+
=== KeyY Generation methid ===
The partitions are represented by tables of DIFI blobs inside a DISA/DIFF structure.
 
The order of the DIFI blobs is the order of the partitions in the image.
 
  
==== DISA ====
+
The [[NCSD]] partition flags determine the method used to generate this keyY.
  
* This is located @ 0x100 in the image, following the MAC header.
+
==== v1 ====
* If the uint32 @ 0x68 in the DISA(the low 8-bits) is non-zero, then the secondary table is is used, otherwise the primary table is used.
 
* If the table has more then 1 DIFI then the uint32 @ 0x168 is the offset from the DATA partition to the file base (masked with 0xFFFFFFFE).
 
  
{| class="wikitable"
+
When all of the flags checked by the running NATIVE_FIRM are clear, the keyY is the following:
 +
{| class="wikitable" border="1"
 
|-
 
|-
! Start
+
! Offset
! Length
+
! Size
! Description
+
! Description
 
|-
 
|-
| 0x00
+
| 0x0
| 4
+
| 0x8
| Magic ("DISA")
+
| First 8-bytes from the plaintext [[NCCH#CXI|CXI]] accessdesc signature.
 
|-
 
|-
| 0x04
+
| 0x8
| 4
+
| 0x4
| Magic Number (0x40000)
+
| u32 CardID0 from [[Gamecards|gamecard]] plaintext-mode command 0x90, Process9 reads this with the [[NTRCARD]] hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.
 
|-
 
|-
| 0x08
+
| 0xC
| 8
+
| 0x4
| Total partition entries in a table
+
| u32 CardID1 from [[Gamecards|gamecard]] plaintext-mode command 0xA0, Process9 reads this with the [[NTRCARD]] hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.
|-
 
| 0x10
 
| 8
 
| Offset to secondary partition table
 
|-
 
| 0x18
 
| 8
 
| Offset to primary partition table
 
|-
 
| 0x20
 
| 8
 
| Partition table size
 
|-
 
| 0x28
 
| 8
 
| SAVE Partition entry offset in the partition table
 
|-
 
| 0x30
 
| 8
 
| SAVE Partition entry length in the partition table
 
|-
 
| 0x38
 
| 8
 
| DATA Partition entry offset in the partition table
 
|-
 
| 0x40
 
| 8
 
| DATA Partition entry length in the partition table
 
|-
 
| 0x48
 
| 8
 
| SAVE Partition offset
 
|-
 
| 0x50
 
| 8
 
| SAVE Partition length
 
|-
 
| 0x58
 
| 8
 
| DATA Partition offset
 
|-
 
| 0x60
 
| 8
 
| DATA Partition length
 
|-
 
| 0x68
 
| 4
 
| Active table (and the offset to the filebase)
 
|-
 
| 0x6C
 
| 0x20
 
| Hash from active table
 
|-
 
| 0x8C
 
| 0x74
 
| Reserved
 
 
|}
 
|}
  
* The hash in the DISA hashes the Active Table (starting from tables's offset to tables's offset + table length) with SHA256.
+
==== v2 ====
* The partition offsets are absolute offsets in the image.
 
* The SAVE partition offset is usually 0x1000. The SAVE/DATA partitions begins with the DPFS partitions, the relative offset for the IVFC partition data is specified by the DPFS header.
 
 
 
The DIFIs table at offset 0x200 in the image has 2 DIFIs when the DATA partition isn't used, 4 DIFIs otherwise. Each partition table contains the SAVE DIFI entry and optionally the DATA entry. The secondary partition table is located at offset 0x200 in the image, and the primary table follows the secondary table.
 
 
 
The non-active table is for backup.
 
 
 
==== DIFF ====
 
  
* This is the [[extdata]] equivalent of DISA, for extdata which use FS. DIFF is only used for extdata.
+
Key Y is the first 0x10 bytes of SHA-256 calculated over the following data
* When the active-table field low 8-bits is non-zero, the secondary partition is used. Otherwise, the primary partition is used.
 
  
{| class="wikitable"
+
{| class="wikitable" border="1"
 
|-
 
|-
! Start
+
! Offset
! Length
+
! Size
! Description
+
! Description
 
|-
 
|-
| 0x00
+
| 0x0
| 4
+
| 0x8
| Magic ("DIFF")
+
| First 8-bytes from the plaintext [[NCCH#CXI|CXI]] accessdesc signature.
 
|-
 
|-
| 0x04
+
| 0x8
| 4
+
| 0x40
| Magic Number (0x30000)
+
| read from a gamecard command(this 0x40-byte data is also read by [[Process_Services_PXI|GetRomId]], which is the gamecard-uniqueID)
|-
 
| 0x08
 
| 8
 
| Secondary partition table offset
 
|-
 
| 0x10
 
| 8
 
| Primary partition table offset
 
|-
 
| 0x18
 
| 8
 
| Partition table length
 
|-
 
| 0x20
 
| 4
 
| Active table (and the offset to the filebase)
 
|-
 
| 0x24
 
| 0x20
 
| Unknown
 
|-
 
| 0x34
 
| 0x20
 
| Hash of the active partition table
 
|-
 
| 0x54
 
| 0xAC
 
| Unknown
 
 
|}
 
|}
  
==== DIFI ====
+
This keyY generation method was implemented with [[2.0.0-2]] via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until [[2.2.0-4]]. The hashed keyY flag[3] implemented with [[2.0.0-2]] was likely never used with retail gamecards.
  
These 0x12C-byte blobs describe the partitions. Following each partition is an unused 0xFFFFFFFF cleartext word in the raw image. Every DIFI blob describes a partition. Partitions are catted together, so after the end of one partition is the beginning of the next.
+
==== v3 ====
  
For most games there's only 1 partition (The SAVE partition) and some (like Asphalt 3D, Steel Diver & Lego Star Wars III) has 2 partitions.
+
[[6.0.0-11]] implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new [[NCSD]] partition flags, all retail games which have the NCSD image finalized after the [[6.0.0-11]] release(and [[6.0.0-11]]+ in the system update partition) will have these flags set for using this new method.
  
* 2 Partitions means that the files inside the SAVE partition is on the DATA partition.
+
First, a SHA-256 hash is calculated over the following data
* The DISA/DIFF headers support a maximum of 2 partitions.
 
  
{| class="wikitable"
+
{| class="wikitable" border="1"
|-
 
! Start
 
! Length
 
! Description
 
|-
 
| 0x00
 
| 4
 
| Magic ("DIFI")
 
|-
 
| 0x04
 
| 4
 
| Magic Number (0x10000)
 
 
|-
 
|-
| 0x08
+
!  Offset
| 8
+
!  Size
| Offset to "IVFC" blob in DIFI (Always 0x44)
+
!  Description
 
|-
 
|-
| 0x10
+
| 0x0
| 8
 
| Size of "IVFC" blob
 
|-
 
| 0x18
 
| 8
 
| Offset to "DPFS" blob in DIFI (Always 0xBC)
 
|-
 
| 0x20
 
| 8
 
| Size of "DPFS" blob
 
|-
 
| 0x28
 
| 8
 
| Offset to the hash in DIFI (Always 0x10C)
 
|-
 
| 0x30
 
| 8
 
| Size of this hash
 
|-
 
| 0x38
 
| 4
 
| Flags (when this byte is non-zero, this is a DATA partition)
 
|-
 
| 0x3C
 
| 8
 
| File base offset (for DATA partitions)
 
|}
 
 
 
==== IVFC ====
 
 
 
{| class="wikitable"
 
|-
 
! Start
 
! Length
 
! Description
 
|-
 
| 0x00
 
| 4
 
| Magic ("IVFC")
 
|-
 
| 0x04
 
| 4
 
| Magic Number (0x20000)
 
|-
 
| 0x08
 
| 0x8
 
| Master hash size
 
|-
 
| 0x10
 
| 0x8
 
| Level 1 relative offset
 
|-
 
| 0x18
 
| 0x8
 
| Level 1 hashdata size
 
|-
 
| 0x20
 
| 0x4
 
| Level 1 block size, in log2
 
|-
 
| 0x24
 
| 0x4
 
| Reserved
 
|-
 
| 0x28
 
 
| 0x8
 
| 0x8
| Level 2 relative offset
+
| First 8-bytes from the plaintext [[NCCH#CXI|CXI]] accessdesc signature.
 
|-
 
|-
| 0x30
 
 
| 0x8
 
| 0x8
| Level 2 hashdata size
 
|-
 
| 0x38
 
| 0x4
 
| Level 2 block size, in log2.
 
|-
 
| 0x3C
 
| 0x4
 
| Reserved
 
|-
 
 
| 0x40
 
| 0x40
| 0x8
+
| Same ID as [[Process_Services_PXI|GetRomId]]
| Level 3 relative offset
 
 
|-
 
|-
 
| 0x48
 
| 0x48
 
| 0x8
 
| 0x8
| Level 3 hashdata size
+
| CXI Program ID
 
|-
 
|-
 
| 0x50
 
| 0x50
| 0x4
 
| Level 3 block size, in log2.
 
|-
 
| 0x54
 
| 0x4
 
| Reserved
 
|-
 
| 0x58
 
| 8
 
| Level 4 filesystem relative offset
 
|-
 
| 0x60
 
| 8
 
| Level 4 filesystem size
 
|-
 
| 0x68
 
| 8
 
| Level 4 filesystem block size, in log2.
 
|-
 
| 0x70
 
| 8
 
| Unknown (usually 0x78=120)
 
|-
 
|}
 
 
* This savegame IVFC is almost identical to the [[RomFS]] IVFC, except for the additional filesystem level. Exactly like RomFS, each level except level4 is a hash-table where each hash entry hashes the data in the next level, padded to the log2 block size.
 
 
==== DPFS ====
 
 
{| class="wikitable"
 
|-
 
! Start
 
! Length
 
! Description
 
|-
 
| 0x00
 
| 4
 
| Magic ("DPFS")
 
|-
 
| 0x04
 
| 4
 
| Magic Number (0x10000)
 
|-
 
| 0x08
 
| 8
 
| Offset to first table
 
|-
 
| 0x10
 
| 8
 
| First table length
 
|-
 
| 0x18
 
| 8
 
| First table block size (1<<value)
 
|-
 
 
| 0x20
 
| 0x20
| 8
+
| ExeFS:/.code hash from the decrypted [[ExeFS]] header
| Offset to second table
 
|-
 
| 0x28
 
| 8
 
| Second table length
 
|-
 
| 0x30
 
| 8
 
| Second table block size (1<<value)
 
|-
 
| 0x38
 
| 8
 
| IVFC partition offset
 
|-
 
| 0x40
 
| 8
 
| IVFC partition size
 
|-
 
| 0x48
 
| 8
 
| IVFC partition block size (1<<value)
 
|-
 
 
|}
 
|}
  
* Every block this table point to is written twice (concatenated). You can see that the offset to the next block is twice the length (except the data which always begin after 0x1000).
+
Then an [[AES]]-CMAC is calculated over this hash. The output CMAC is used for keyY. The key slot for this CMAC is 0x2F.
* The offsets contained in the DPFS and IVFC are relative to the partition offset in the DISA/DIFF. The offsets from the IVFC are additionally added with the IVFC partition offset from the DPFS.
 
  
The first partition's data usually starts at 0x2000. First comes the hashtable (usually start @ 0x40 into the partition) and then the filesystem.
+
The 0x2F keyY used for calculating this AES-CMAC (not to be confused with the final keyY for decrypting/signing savegames) is initialized while NATIVE_FIRM is loading, this keyY is generated via the [[RSA]] engine. The RSA slot used here is slot0(key-data for slot0 is initialized by bootrom), this RSA slot0 key-data is overwritten during system boot. This RSA slot0 key-data gets overwritten with the RSA key-data used for verifying RSA signatures, every time Process9 verifies any RSA signatures except for [[NCCH|NCCH]] accessdesc signatures. Starting with [[7.0.0-13]] this key-init function used at boot is also used to initialize a separate keyslot used for the new [[NCCH]] encryption method.
  
The hashtable entries' size is 2^x where x is the 'Filesystem block size' from the IVFC block.
+
This [[FIRM|Process9]] key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.
  
'''DIFI Hash'''
+
== Gamecard wear leveling ==
  
The last 0x20-bytes of the partition following the DIFI, IVFC and DPFS is a SHA256 hash. The offset to this hash is stored in the DIFI. This hashes the IVFC level 1, with the buffer which is hashed aligned to the IVFC level 1 log2 block-size.
+
The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.
  
'''Summary Drawing'''
+
There are two versions of wear leveling have been observed. V1 is used for 128KB and 512 KB CARD1 flash chips. V2 is used for 1MB CARD1 flash chips (uncommon. Pokemon Sun/Moon is an exampe).
  
[[File:Sfimg_drawing.png]]
+
First, there are two 32-bit integers whose purposes are currently unknown. They generally increase the value as the savegame is written more times, so probably counter for how many times the journal became full and got flushed into the block map, and/or how many times <code>alloc_cnt</code> has wrapped around.  
  
==== The SAVE partition ====
+
Then comes the actual blockmap. The block map contains entries of 10 bytes (V1) or 2 bytes (V2) with total number of <code>(flash_size / 0x1000 - 1)</code>.
 +
The blockmap entry is simple:
 +
<pre>
 +
struct blockmap_entry_v1 {
 +
        uint8_t phys_sec; // when bit7 is set, block is initialized and has checksums, otherwise checksums are all zero
 +
        uint8_t alloc_cnt;
 +
        uint8_t chksums[8];
 +
} __attribute__((__packed__));
  
* The SAVE filesystem works with a backup. There are two SAVE blocks inside the partition concatenated. Which SAVE block is the updated one is unknown yet.. (I'm guessing from experience that (image[0x100B] & 0x20) == 0x20 --> 1st SAVE --[[User:Elisherer|Elisherer]] 01:30, 18 October 2011 (CEST))
+
struct blockmap_entry_v2 {
 +
        // Note that the phys_sec and alloc_cnt field are swapped in v2,
 +
        // but the initialized bit is still on the first byte
 +
        uint8_t alloc_cnt; // when bit7 is set, block is initialized
 +
        uint8_t phys_sec;
 +
        // v2 has no chksums
 +
} __attribute__((__packed__));
 +
</pre>
  
'''Finding the folders table:'''
+
There's one entry per 0x1000-byte sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).
* If DATA partition exists: At folder table exact offset from the SAVE struct (from the beginning of the struct).
 
* Otherwise: The 'folder table offset' * 'folder table media' (=0x200) from the 'filestore offset'. (usually 0 from filebase)
 
  
'''Finding the files table:'''
+
A 2-byte CRC16 follows the block map. For V1 it immediately follows the last block map entry. For V2 it is located at 0x3FE, and bytes before the CRC is padded with zero. The CRC16 checks all the bytes before it, including the two unknown integers, the block map, and the padding bytes for V2. The CRC standard used looks like CRC-16-IBM (modbus). Here is the code in Rust for it
* If DATA partition exists: At file table exact offset from the SAVE struct (from the beginning of the struct).
 
* Otherwise: The 'file table offset' * 'file table media' (=0x200) from the 'filestore offset'.
 
  
'''Detemining the filestore base:'''
 
* If DATA partition exists: At file base from the DATA's DIFI struct into the DATA partition.
 
* Otherwise: At the 'filestore offset' from the beginning of the SAVE struct.
 
 
Folder's entry structure:
 
 
<pre>
 
<pre>
struct folder_entry {
+
fn crc16(data: &[u8]) -> u16 {
    u32 parent_folder_index;
+
    let poly = 0xA001;
    u8  filename[0x10];
+
    let mut crc = 0xFFFFu16;
    u32 folder_index;
+
    for byte in data {
    u32 unk1;  
+
        crc ^= <u16>::from(*byte);
    u32 last_file_index;
+
        for _ in 0..8 {
    u32 unk3;  
+
            let b = crc & 1 != 0;
    u32 unk4;
+
            crc >>= 1;
}
+
            if b {
 +
                crc ^= poly;
 +
            }
 +
        }
 +
    }
 +
    crc
 +
}
 
</pre>
 
</pre>
  
File's entry structure:
+
Then comes the journal. The journal contains entries that describes how sectors should be remapped. The rest bytes before 0x1000 after all journal entries are padded with 0xFF
 +
The journal entry structure is as follows:
 
<pre>
 
<pre>
struct file_entry {
+
struct journal_entry_half {
    u32 parent_folder_index;
+
        uint8_t virt_sec;       // Mapped to sector
    u8 filename[0x10];
+
        uint8_t prev_virt_sec; // Physical sector previously mapped to
    u32 index;
+
        uint8_t phys_sec;       // Mapped from sector
    u32 unk1; // magic?
+
        uint8_t prev_phys_sec; // Virtual sector previously mapped to
    u32 block_offset;
+
        uint8_t phys_realloc_cnt;       // Amount of times physical sector has been remapped
    u64 file_size;
+
        uint8_t virt_realloc_cnt;       // Amount of times virtual sector has been remapped
    u32 unk2; // flags?
+
        uint8_t chksums[8];     // Unused & uninitialized for V2
    u32 unk3;
+
} __attribute__((__packed__));
}
 
</pre>
 
  
The first entry in both tables is the count of the table, the parent directory index will be the amount of table rows. The root includes itself, so there are the amount - 1 (minus one) folders in the root directory (or files). The entries that follow after the root are the actual folders/files.
+
struct journal_entry{
 
+
        struct journal_entry_half entry;
Reading the files out is as simple as taking the file base offset and adding (block_offset * 0x200) to it.
+
        struct journal_entry_half dupe; // same data as `entry`. No idea what this is used fore
 
+
        uint32_t uninitialized;        // 0xFFFFFFFF in newer system
Here's a follow-up example from the Legend of Zelda: Ocarina of Time 3D:
+
}__attribute__((__packed__));
<pre>
 
//FST entry = SAVE base + File base + (FST offset * 0x200) + (FST entry # * 0x30)
 
//0x2600    = 0x2000    + 0x400    + (0x1        * 0x200) + (0x0        * 0x30)
 
 
 
00002600: 03000000 09000000 00000000 00000000  ................
 
00002610: 00000000 00000000 00000000 00000000  ................
 
00002620: 00000000 00000000 00000000 00000000  ................
 
00002630: 01000000 73797374 656D2E64 61740000  ....system.dat..
 
00002640: 00000000 00000000 D57B1100 02000000  ........Õ{......
 
00002650: 22000000 00000000 E8121500 00000000  ".......è.......
 
00002660: 01000000 73617665 30302E62 696E0000  ....save00.bin..
 
00002670: 00000000 01000000 69921100 03000000  ........i’......
 
00002680: DC140000 00000000 04000000 00000000  Ü...............
 
 
</pre>
 
</pre>
  
{| class="wikitable"
 
|-
 
! Start
 
! Length
 
! Description
 
|-
 
| 0x00
 
| 4
 
| Magic ("SAVE")
 
|-
 
| 0x04
 
| 4
 
| Magic Number (0x40000)
 
|-
 
| 0x08
 
| 8
 
| Offset to data in this SAVE header(normally 0x20)
 
|-
 
| 0x10
 
| 8
 
| Partition Size [medias]
 
|-
 
| 0x18
 
| 4
 
| Partition Media Size
 
|-
 
| 0x1C
 
| 8
 
| Unknown
 
|-
 
| 0x24
 
| 4
 
| Unknown
 
|-
 
| 0x28
 
| 8
 
| FolderMap Offset
 
|-
 
| 0x30
 
| 4
 
| FolderMap Size
 
|-
 
| 0x34
 
| 4
 
| FolderMap Media Size
 
|-
 
| 0x38
 
| 8
 
| FileMap Offset
 
|-
 
| 0x40
 
| 4
 
| FileMap Size
 
|-
 
| 0x44
 
| 4
 
| FileMap Media Size
 
|-
 
| 0x48
 
| 8
 
| BlockMap Offset
 
|-
 
| 0x50
 
| 4
 
| BlockMap Size
 
|-
 
| 0x54
 
| 4
 
| BlockMap Media Size
 
|-
 
| 0x58
 
| 8
 
| File store offset (from SAVE)
 
|-
 
| 0x60
 
| 4
 
| File store length [medias]
 
|-
 
| 0x64
 
| 4
 
| File store media size
 
|-
 
| 0x68
 
| 4/8
 
| Folders Table offset (8 bytes in DATA)
 
|-
 
| 0x6C
 
| 4
 
| Folders Table Length (medias) (Only in no DATA)
 
|-
 
| 0x70
 
| 4
 
| Folders Table unknown
 
|-
 
| 0x74
 
| 4
 
| Folders Table Media size
 
|-
 
| 0x78
 
| 4/8
 
| Files Table offset (8 bytes in DATA)
 
|-
 
| 0x7C
 
| 4
 
| Files Table Length (medias) (Only in no DATA)
 
|-
 
| 0x80
 
| 4
 
| Files Table unknown
 
|-
 
| 0x84
 
| 4
 
| Files Table Media size
 
|-
 
|}
 
 
* The FolderMap and FileMap still unknown. They are tables of uint32.
 
* The BlockMap is a map of the blocks in the filestore. An entry in the BlockMap is 2 uint32: {uint32 start_block; uint32 end_block; }. This is still being researched. (You can use [[3DSExplorer]] to see those maps.
 
  
'''Summary Drawing'''
+
The checksums in the blockmap/journal entries work as follows:
 +
* each byte is the checksum of an encrypted 0x200 bytes large block
 +
* to calculate the checksum, a CRC16 of the block (same CRC16 algorithm as above) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum
  
[[File:Sfsave_drawing.png]]
+
== Initialization ==
 
 
=== Initialization ===
 
  
 
When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.
 
When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.
Line 695: Line 216:
 
I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
 
I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
  
=== Fun Facts ===
+
== Fun Facts ==
  
 
If you have facts that you found out by looking at the binary files please share them here:
 
If you have facts that you found out by looking at the binary files please share them here:
Line 701: Line 222:
 
* From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
 
* From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
  
=== Tools ===
+
== Tools ==
  
* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames, however it currently does not write the modified save image to the input savegame file.
+
* [https://github.com/wwylele/save3ds save3ds] supports reading and modifying savegames, extdata and title database in FUSE filesystem or batch extracting/importing.
 +
* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard. (This is an old tool that doesn't handle the savegame format correctly. --[[User:Wwylele|Wwylele]] ([[User talk:Wwylele|talk]]) 16:13, 2 December 2019 (CET))
 
* [[3DSExplorer]] supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).
 
* [[3DSExplorer]] supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).
 +
* [https://github.com/wwylele/3ds-save-tool wwylele's 3ds-save-tool] supports extracting files from savegames and extdata. It properly reconstructs data from the DPFS tree and extracts files in directories hierarchy.
 +
 
[[セーブデータ|Japanese]]
 
[[セーブデータ|Japanese]]

Latest revision as of 05:51, 23 December 2019

This page describes the format and encryption of savegames contained in gamecards, SD and NAND. You can find savegames from various 3DS games on the Games page.

Overview[edit]

Savegames are stored in DISA container format. Inside the DISA container, it forms a FAT filesystem. Please refer to these pages for how to fully extract save files. This page only describes additional encryption wear leveling on top of the DISA container. These layers only apply to gamecard save games. SD savegames and NAND savegames are DISA containers in plaintext after decrypting the common SD/NAND encryption layer.

Gamecard savegame Encryption[edit]

Gamecard encryption is AES-CTR applied on top of DISA container, but below the wear leveling layer (if exists). The same key Y used for encryption is also used for DISA CMAC signing. Several versions of encryption scheme have been introduced over the time.

FW Introduced Old3DS AES Keyslots (Encryotion / CMAC) KeyY generation method Repeating CTR
The initial version Yes 0x37 / 0x33 v1 Yes
2.0.0-2 Yes 0x37 / 0x33 v2 Yes
2.2.0-4 Yes 0x37 / 0x33 v2 No
6.0.0-11 Yes 0x37 / 0x33 v3 No
9.6.0-X No 0x1A / 0x19 v2? No

Repeating CTR Fail[edit]

On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.

The reason this works is because the stream cipher used has a period of 512 bytes. That is to say, it will repeat the same keystream after 512 bytes. The way you encrypt with a stream cipher is you XOR your data with the keystream as it is produced. Unfortunately, if your streamcipher repeats and you are encrypting a known plain-text (in our case, zeros) you are basically giving away your valuable keystream.

So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.


KeyY Generation methid[edit]

The NCSD partition flags determine the method used to generate this keyY.

v1[edit]

When all of the flags checked by the running NATIVE_FIRM are clear, the keyY is the following:

Offset Size Description
0x0 0x8 First 8-bytes from the plaintext CXI accessdesc signature.
0x8 0x4 u32 CardID0 from gamecard plaintext-mode command 0x90, Process9 reads this with the NTRCARD hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.
0xC 0x4 u32 CardID1 from gamecard plaintext-mode command 0xA0, Process9 reads this with the NTRCARD hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.

v2[edit]

Key Y is the first 0x10 bytes of SHA-256 calculated over the following data

Offset Size Description
0x0 0x8 First 8-bytes from the plaintext CXI accessdesc signature.
0x8 0x40 read from a gamecard command(this 0x40-byte data is also read by GetRomId, which is the gamecard-uniqueID)

This keyY generation method was implemented with 2.0.0-2 via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until 2.2.0-4. The hashed keyY flag[3] implemented with 2.0.0-2 was likely never used with retail gamecards.

v3[edit]

6.0.0-11 implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new NCSD partition flags, all retail games which have the NCSD image finalized after the 6.0.0-11 release(and 6.0.0-11+ in the system update partition) will have these flags set for using this new method.

First, a SHA-256 hash is calculated over the following data

Offset Size Description
0x0 0x8 First 8-bytes from the plaintext CXI accessdesc signature.
0x8 0x40 Same ID as GetRomId
0x48 0x8 CXI Program ID
0x50 0x20 ExeFS:/.code hash from the decrypted ExeFS header

Then an AES-CMAC is calculated over this hash. The output CMAC is used for keyY. The key slot for this CMAC is 0x2F.

The 0x2F keyY used for calculating this AES-CMAC (not to be confused with the final keyY for decrypting/signing savegames) is initialized while NATIVE_FIRM is loading, this keyY is generated via the RSA engine. The RSA slot used here is slot0(key-data for slot0 is initialized by bootrom), this RSA slot0 key-data is overwritten during system boot. This RSA slot0 key-data gets overwritten with the RSA key-data used for verifying RSA signatures, every time Process9 verifies any RSA signatures except for NCCH accessdesc signatures. Starting with 7.0.0-13 this key-init function used at boot is also used to initialize a separate keyslot used for the new NCCH encryption method.

This Process9 key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.

Gamecard wear leveling[edit]

The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.

There are two versions of wear leveling have been observed. V1 is used for 128KB and 512 KB CARD1 flash chips. V2 is used for 1MB CARD1 flash chips (uncommon. Pokemon Sun/Moon is an exampe).

First, there are two 32-bit integers whose purposes are currently unknown. They generally increase the value as the savegame is written more times, so probably counter for how many times the journal became full and got flushed into the block map, and/or how many times alloc_cnt has wrapped around.

Then comes the actual blockmap. The block map contains entries of 10 bytes (V1) or 2 bytes (V2) with total number of (flash_size / 0x1000 - 1). The blockmap entry is simple:

struct blockmap_entry_v1 {
        uint8_t phys_sec; // when bit7 is set, block is initialized and has checksums, otherwise checksums are all zero
        uint8_t alloc_cnt;
        uint8_t chksums[8];
} __attribute__((__packed__));

struct blockmap_entry_v2 {
        // Note that the phys_sec and alloc_cnt field are swapped in v2, 
        // but the initialized bit is still on the first byte
        uint8_t alloc_cnt; // when bit7 is set, block is initialized
        uint8_t phys_sec; 
        // v2 has no chksums
} __attribute__((__packed__));

There's one entry per 0x1000-byte sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).

A 2-byte CRC16 follows the block map. For V1 it immediately follows the last block map entry. For V2 it is located at 0x3FE, and bytes before the CRC is padded with zero. The CRC16 checks all the bytes before it, including the two unknown integers, the block map, and the padding bytes for V2. The CRC standard used looks like CRC-16-IBM (modbus). Here is the code in Rust for it

fn crc16(data: &[u8]) -> u16 {
    let poly = 0xA001;
    let mut crc = 0xFFFFu16;
    for byte in data {
        crc ^= <u16>::from(*byte);
        for _ in 0..8 {
            let b = crc & 1 != 0;
            crc >>= 1;
            if b {
                crc ^= poly;
            }
        }
    }
    crc
}

Then comes the journal. The journal contains entries that describes how sectors should be remapped. The rest bytes before 0x1000 after all journal entries are padded with 0xFF The journal entry structure is as follows:

struct journal_entry_half {
        uint8_t virt_sec;       // Mapped to sector
        uint8_t prev_virt_sec;  // Physical sector previously mapped to
        uint8_t phys_sec;       // Mapped from sector
        uint8_t prev_phys_sec;  // Virtual sector previously mapped to
        uint8_t phys_realloc_cnt;       // Amount of times physical sector has been remapped
        uint8_t virt_realloc_cnt;       // Amount of times virtual sector has been remapped
        uint8_t chksums[8];     // Unused & uninitialized for V2
} __attribute__((__packed__));

struct journal_entry{
        struct journal_entry_half entry;
        struct journal_entry_half dupe; // same data as `entry`. No idea what this is used fore
        uint32_t uninitialized;         // 0xFFFFFFFF in newer system
}__attribute__((__packed__));


The checksums in the blockmap/journal entries work as follows:

  • each byte is the checksum of an encrypted 0x200 bytes large block
  • to calculate the checksum, a CRC16 of the block (same CRC16 algorithm as above) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum

Initialization[edit]

When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.

I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --Elisherer 22:41, 15 October 2011 (CEST)

Fun Facts[edit]

If you have facts that you found out by looking at the binary files please share them here:

  • From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --Elisherer 22:41, 15 October 2011 (CEST)

Tools[edit]

  • save3ds supports reading and modifying savegames, extdata and title database in FUSE filesystem or batch extracting/importing.
  • 3dsfuse supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard. (This is an old tool that doesn't handle the savegame format correctly. --Wwylele (talk) 16:13, 2 December 2019 (CET))
  • 3DSExplorer supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).
  • wwylele's 3ds-save-tool supports extracting files from savegames and extdata. It properly reconstructs data from the DPFS tree and extracts files in directories hierarchy.

Japanese