3dbrew

Savegames: Difference between revisions

← Older edit
Wwylele (talk | contribs)
242 edits
Nuke the old content. Sorry for removing the next diagram, but it is really outdated. I will make a new one but please fix wiki image uploading first!
m fix typos
 
(13 intermediate revisions by one other user not shown)
Line 2: Line 2:


== Overview ==
== Overview ==
Savegames are stored in [[DISA and DIFF|DISA container formats]](follow this link for the container format description). It forms a file system inside the inner content of the container. In this page only the inner file system format of the content is described.
Savegames are stored in [[DISA and DIFF|DISA container format]]. Inside the DISA container, it forms a [[Inner FAT|FAT filesystem]]. '''Please refer to these pages for how to fully extract save files'''. This page only describes additional encryption wear leveling on top of the DISA container. These layers only apply to gamecard save games. SD savegames and NAND savegames are DISA containers in plaintext after decrypting the common SD/NAND encryption layer.


Unlike SD and NAND savegames, gamecard savegames has additional encryption + wear leveling layer. They are described in the following sections.
== Gamecard savegame Encryption ==


== Gamecard savegame Encryption ===
Gamecard encryption is AES-CTR applied on top of DISA container, but below the wear leveling layer (if exists). The same key Y used for encryption is also used for DISA CMAC signing. Several versions of encryption scheme have been introduced over the time.
 
{| class="wikitable" border="1"
|-
!  FW Introduced
!  Old3DS
!  [[AES#Keyslot|AES Keyslots]] (Encryption / CMAC)
!  KeyY generation method
!  Repeating CTR
|-
| The initial version
| style="background: #ccffbb" | Yes
| 0x37 / 0x33
| v1
| style="background: #ccffbb" | Yes
|-
| [[2.0.0-2]]
| style="background: #ccffbb" | Yes
| 0x37 / 0x33
| v2
| style="background: #ccffbb" | Yes
|-
| [[2.2.0-4]]
| style="background: #ccffbb" | Yes
| 0x37 / 0x33
| v2
| style="background: #ffccbb" | No
|-
| [[6.0.0-11]]
| style="background: #ccffbb" | Yes
| 0x37 / 0x33
| v3
| style="background: #ffccbb" | No
|-
| [[9.6.0-24|9.6.0-X]]
| style="background: #ffccbb" | No
| 0x1A / 0x19
| v2?
| style="background: #ffccbb" | No
|}


=== Repeating CTR Fail ===
=== Repeating CTR Fail ===
Line 15: Line 54:
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.


=== Savegame keyY ===


All gamecard and SD savegames are encrypted with AES-CTR. The base CTR for gamecard savegames is all-zero. The gamecard savegame [[AES|keyslots]]' keyY(these savegame keyslots use the hardware key-generator) is unique for each region and for each game. The [[NCSD]] partition flags determine the method used to generate this keyY. When the save [[NCSD]] flags checked by the running NATIVE_FIRM are all-zero, the system will use the repeating CTR, otherwise a proper CTR which never repeats within the image is used.
=== KeyY Generation method ===
 
The [[NCSD]] partition flags determine the method used to generate this keyY.


The [[AES]]-CMAC (which uses a hardware key-generator keyslot, as mentioned above) at the the beginning of the savegame must match the calculated CMAC using the DISA/DIFF data, otherwise the savegame is considered corrupted(see below).
==== v1 ====


When all of the flags checked by the running NATIVE_FIRM are clear, the keyY(original keyY method used with saves where the CTR repeats within the image) is the following:
When all of the flags checked by the running NATIVE_FIRM are clear, the keyY is the following:
{| class="wikitable" border="1"
{| class="wikitable" border="1"
|-
|-
Line 41: Line 81:
|}
|}


==== [[2.0.0-2]] Hashed keyY and [[2.2.0-4]] Savegame Encryption ====
==== v2 ====


When certain [[NCSD]] partition flags are set, a SHA-256 hash is calculated over the data from the CXI(same data used with the original plain keyY), and the 0x40-bytes read from a gamecard command(this 0x40-byte data is also read by [[Process_Services_PXI|GetRomId]], which is the gamecard-uniqueID). The first 0x10-bytes from this hash is used for the keyY. When flag[7] is set, the CTR will never repeat within the save image, unlike the original CTR-method. All games which had the retail NCSD image finalized after the [[2.2.0-4]] update(and contain [[2.2.0-4]]+ in the [[System Update CFA|System update partition]]), use this encryption method.
Key Y is the first 0x10 bytes of SHA-256 calculated over the following data
 
{| class="wikitable" border="1"
|-
!  Offset
!  Size
!  Description
|-
| 0x0
| 0x8
| First 8-bytes from the plaintext [[NCCH#CXI|CXI]] accessdesc signature.
|-
| 0x8
| 0x40
| read from a gamecard command(this 0x40-byte data is also read by [[Process_Services_PXI|GetRomId]], which is the gamecard-uniqueID)
|}


This keyY generation method was implemented with [[2.0.0-2]] via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until [[2.2.0-4]]. The hashed keyY flag[3] implemented with [[2.0.0-2]] was likely never used with retail gamecards.
This keyY generation method was implemented with [[2.0.0-2]] via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until [[2.2.0-4]]. The hashed keyY flag[3] implemented with [[2.0.0-2]] was likely never used with retail gamecards.


==== [[6.0.0-11]] Savegame keyY ====
==== v3 ====


[[6.0.0-11]] implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new [[NCSD]] partition flags, all retail games which have the NCSD image finalized after the [[6.0.0-11]] release(and [[6.0.0-11]]+ in the system update partition) will have these flags set for using this new method.
[[6.0.0-11]] implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new [[NCSD]] partition flags, all retail games which have the NCSD image finalized after the [[6.0.0-11]] release(and [[6.0.0-11]]+ in the system update partition) will have these flags set for using this new method.


A SHA-256 hash is calculated over the same data used with the above hashed keyY method, after hashing the above data the following data is hashed: the CXI programID, and the ExeFS:/.code hash from the decrypted [[ExeFS]] header. An [[AES]]-CMAC (the keyslot used for this uses the hardware key-scrambler) is then calculated over this hash, the output CMAC is used for the savegame keyY.
First, a SHA-256 hash is calculated over the following data
 
{| class="wikitable" border="1"
|-
!  Offset
!  Size
!  Description
|-
| 0x0
| 0x8
| First 8-bytes from the plaintext [[NCCH#CXI|CXI]] accessdesc signature.
|-
| 0x8
| 0x40
| Same ID as [[Process_Services_PXI|GetRomId]]
|-
| 0x48
| 0x8
| CXI Program ID
|-
| 0x50
| 0x20
| ExeFS:/.code hash from the decrypted [[ExeFS]] header
|}
 
Then an [[AES]]-CMAC is calculated over this hash. The output CMAC is used for keyY. The key slot for this CMAC is 0x2F.


The keyY used for calculating this AES-CMAC is initialized while NATIVE_FIRM is loading, this keyY is generated via the [[RSA]] engine. The RSA slot used here is slot0(key-data for slot0 is initialized by bootrom), this RSA slot0 key-data is overwritten during system boot. This RSA slot0 key-data gets overwritten with the RSA key-data used for verifying RSA signatures, every time Process9 verifies any RSA signatures except for [[NCCH|NCCH]] accessdesc signatures. Starting with [[7.0.0-13]] this key-init function used at boot is also used to initialize a separate keyslot used for the new [[NCCH]] encryption method.
The 0x2F keyY used for calculating this AES-CMAC (not to be confused with the final keyY for decrypting/signing savegames) is initialized while NATIVE_FIRM is loading, this keyY is generated via the [[RSA]] engine. The RSA slot used here is slot0(key-data for slot0 is initialized by bootrom), this RSA slot0 key-data is overwritten during system boot. This RSA slot0 key-data gets overwritten with the RSA key-data used for verifying RSA signatures, every time Process9 verifies any RSA signatures except for [[NCCH|NCCH]] accessdesc signatures. Starting with [[7.0.0-13]] this key-init function used at boot is also used to initialize a separate keyslot used for the new [[NCCH]] encryption method.


This [[FIRM|Process9]] key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.
This [[FIRM|Process9]] key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.
Line 61: Line 141:
The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.
The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.


First, there are 8 bytes whose purposes are currently unknown. Then comes the actual blockmap.
There are two versions of wear leveling have been observed. V1 is used for 128KB and 512 KB CARD1 flash chips. V2 is used for 1MB CARD1 flash chips (uncommon. Pokemon Sun/Moon is an example).
The blockmap structure is simple:
 
First, there are two 32-bit integers whose purposes are currently unknown. They generally increase the value as the savegame is written more times, so probably counter for how many times the journal became full and got flushed into the block map, and/or how many times <code>alloc_cnt</code> has wrapped around.
 
Then comes the actual blockmap. The block map contains entries of 10 bytes (V1) or 2 bytes (V2) with total number of <code>(flash_size / 0x1000 - 1)</code>.  
The blockmap entry is simple:
<pre>
<pre>
struct header_entry {
struct blockmap_entry_v1 {
         uint8_t phys_sec; // when bit7 is set, block has checksums, otherwise checksums are all zero
         uint8_t phys_sec; // when bit7 is set, block is initialized and has checksums, otherwise checksums are all zero
         uint8_t alloc_cnt;
         uint8_t alloc_cnt;
         uint8_t chksums[8];
         uint8_t chksums[8];
} __attribute__((__packed__));
struct blockmap_entry_v2 {
        // Note that the phys_sec and alloc_cnt field are swapped in v2,
        // but the initialized bit is still on the first byte
        uint8_t alloc_cnt; // when bit7 is set, block is initialized
        uint8_t phys_sec;
        // v2 has no chksums
} __attribute__((__packed__));
} __attribute__((__packed__));
</pre>
</pre>


There's one entry per sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).
There's one entry per 0x1000-byte sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).


The 2 bytes that follow the blockmap are the CRC16 (with starting value 0xFFFF (like modbus)) of the first 8 bytes and the blockmap.
A 2-byte CRC16 follows the block map. For V1 it immediately follows the last block map entry. For V2 it is located at 0x3FE, and bytes before the CRC is padded with zero. The CRC16 checks all the bytes before it, including the two unknown integers, the block map, and the padding bytes for V2. The CRC standard used looks like CRC-16-IBM (modbus). Here is the code in Rust for it


Then comes the journal.
The journal structure is as follows:
<pre>
<pre>
struct sector_entry {
fn crc16(data: &[u8]) -> u16 {
    let poly = 0xA001;
    let mut crc = 0xFFFFu16;
    for byte in data {
        crc ^= <u16>::from(*byte);
        for _ in 0..8 {
            let b = crc & 1 != 0;
            crc >>= 1;
            if b {
                crc ^= poly;
            }
        }
    }
    crc
}
</pre>
 
Then comes the journal. The journal contains entries that describes how sectors should be remapped. The rest bytes before 0x1000 after all journal entries are padded with 0xFF
The journal entry structure is as follows:
<pre>
struct journal_entry_half {
         uint8_t virt_sec;      // Mapped to sector
         uint8_t virt_sec;      // Mapped to sector
         uint8_t prev_virt_sec;  // Physical sector previously mapped to
         uint8_t prev_virt_sec;  // Physical sector previously mapped to
Line 85: Line 195:
         uint8_t phys_realloc_cnt;      // Amount of times physical sector has been remapped
         uint8_t phys_realloc_cnt;      // Amount of times physical sector has been remapped
         uint8_t virt_realloc_cnt;      // Amount of times virtual sector has been remapped
         uint8_t virt_realloc_cnt;      // Amount of times virtual sector has been remapped
         uint8_t chksums[8];
         uint8_t chksums[8];     // Unused & uninitialized for V2
} __attribute__((__packed__));
} __attribute__((__packed__));


struct long_sector_entry{
struct journal_entry{
         struct sector_entry sector;
         struct journal_entry_half entry;
         struct sector_entry dupe;
         struct journal_entry_half dupe; // same data as `entry`. No idea what this is used fore
         uint32_t magic;
         uint32_t uninitialized;         // 0xFFFFFFFF in newer system
}__attribute__((__packed__));
}__attribute__((__packed__));
</pre>
</pre>


With magic being a constant 0x080d6ce0.


The checksums in the blockmap/journal entries work as follows:
The checksums in the blockmap/journal entries work as follows:
* each byte is the checksum of an encrypted 0x200 bytes large block
* each byte is the checksum of an encrypted 0x200 bytes large block
* to calculate the checksum, a CRC16 of the block (with starting value 0xFFFF) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum
* to calculate the checksum, a CRC16 of the block (same CRC16 algorithm as above) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum
 
== Partitions ==
 
A savegame can have one or two partitions (specified by the [[DISA and DIFF|DISA container format]]). The first partition is called the SAVE partition, and the second optional partition is called the DATA partition.
 
=== The SAVE partition ===
 
'''Finding the folders table:'''
* If DATA partition exists: At folder table exact offset from the SAVE struct (from the beginning of the struct).
* Otherwise: The 'folder table offset' * 'folder table media' (=0x200) from the 'filestore offset'. (usually 0 from filebase)
 
'''Finding the files table:'''
* If DATA partition exists: At file table exact offset from the SAVE struct (from the beginning of the struct).
* Otherwise: The 'file table offset' * 'file table media' (=0x200) from the 'filestore offset'.
 
'''Detemining the filestore base:'''
* If DATA partition exists: At file base from the DATA's DIFI struct into the DATA partition.
* Otherwise: At the 'filestore offset' from the beginning of the SAVE struct.
 
Folder's entry structure:
<pre>
struct folder_entry {
    u32 parent_folder_index;    // 0 for the root folder (the second entry)
    u8  filename[0x10];          // "" for the root folder (the second entry)
    u32 next_folder_index;      // 0 if the last folder
    u32 first_sub_folder_index;  // 0 if has no sub folder
    u32 first_sub_file_index;    // 0 if has no sub file(?)
    u32 unk1;                    // flags? always 0?
    u32 unk2;                    // the index of the next folder with the same path hash? 0 if the end?
}
struct dummy_folder_entry {
    u32 total_entry_count;
    u32 unk;                    // max_entry_count? FolderMapSize related?
    u32 zero_padding[7];
    u32 next_dummy_folder_index; // 0 if the last dummy folder;
}
</pre>
 
File's entry structure:
<pre>
struct file_entry {
    u32 parent_folder_index;
    u8  filename[0x10];
    u32 next_file_index;      // 0 if the last file
    u32 unk1;                  // looks like time stamp?
    u32 block_offset;
    u64 file_size;
    u32 unk2;                  // flags?
    u32 unk3;                  // the index of the next file with the same path hash? 0 if the end?
}
struct dummy_file_entry {
    u32 total_entry_count;
    u32 unk;                  // max_entry_count? FileMapSize related?
    u32 zero_padding[9];
    u32 next_dummy_file_index; // 0 if the last dummy file;
}
</pre>
 
Both table contains some "dummy" entries, which are probably left there when deleting files/folders, reserved for future use. The first entry of the two tables are always a dummy entry (as the "first" dummy entry).
 
Reading the files out is as simple as taking the file base offset and adding (block_offset * 0x200) to it.
 
Here's a follow-up example from the Legend of Zelda: Ocarina of Time 3D:
<pre>
//FST entry = SAVE base + File base + (FST offset * 0x200) + (FST entry # * 0x30)
//0x2600    = 0x2000    + 0x400    + (0x1        * 0x200) + (0x0        * 0x30)
 
00002600: 03000000 09000000 00000000 00000000  ................
00002610: 00000000 00000000 00000000 00000000  ................
00002620: 00000000 00000000 00000000 00000000  ................
00002630: 01000000 73797374 656D2E64 61740000  ....system.dat..
00002640: 00000000 00000000 D57B1100 02000000  ........Õ{......
00002650: 22000000 00000000 E8121500 00000000  ".......è.......
00002660: 01000000 73617665 30302E62 696E0000  ....save00.bin..
00002670: 00000000 01000000 69921100 03000000  ........i’......
00002680: DC140000 00000000 04000000 00000000  Ü...............
</pre>
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("SAVE")
|-
| 0x04
| 4
| Magic Number (0x40000)
|-
| 0x08
| 8
| Offset to data in this SAVE header(normally 0x20)
|-
| 0x10
| 8
| Partition Size [medias]
|-
| 0x18
| 4
| Partition Media Size
|-
| 0x1C
| 8
| Unknown
|-
| 0x24
| 4
| Media-size for the below sections
|-
| 0x28
| 8
| Folder Hash Table Offset
|-
| 0x30
| 4
| Folder Hash Table Size / Bucket Count
|-
| 0x34
| 4
| Unknown, Folder Hash Table size-related
|-
| 0x38
| 8
| File Hash Table Offset
|-
| 0x40
| 4
| File Hash Table Size / Bucket Count
|-
| 0x44
| 4
| Unknown, File Hash Table size-related
|-
| 0x48
| 8
| BlockMap Offset
|-
| 0x50
| 4
| BlockMap Size
|-
| 0x54
| 4
| Unknown, BlockMap size-related
|-
| 0x58
| 8
| File store offset (from SAVE)
|-
| 0x60
| 4
| File store length [medias]
|-
| 0x64
| 4
| Unknown, File store size-related
|-
| 0x68
| 4/8
| Folders Table offset (8 bytes in DATA)
|-
| 0x6C
| 4
| Folders Table Length (medias) (Only in no DATA)
|-
| 0x70
| 4
| Maximum Folder Count
|-
| 0x74
| 4
| Unknown, Folders Table size-related
|-
| 0x78
| 4/8
| Files Table offset (8 bytes in DATA)
|-
| 0x7C
| 4
| Files Table Length (medias) (Only in no DATA)
|-
| 0x80
| 4
| Maximum File Count
|-
| 0x84
| 4
| Unknown, Files Table size-related
|-
|}
 
* The folder and file hash table are tables of uint32. They are probably for quick lookup and are similar to those in [[RomFS]]
* The BlockMap is a map of the blocks in the filestore. An entry in the BlockMap is 2 uint32: {uint32 start_block; uint32 end_block; }. This is still being researched. (You can use [[3DSExplorer]] to see those maps.
 
'''Summary Drawing'''
 
[[File:Sfsave_drawing.png]]


== Initialization ==
== Initialization ==
Line 315: Line 224:
== Tools ==
== Tools ==


* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard.
* [https://github.com/wwylele/save3ds save3ds] supports reading and modifying savegames, extdata and title database in FUSE filesystem or batch extracting/importing.
* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard. (This is an old tool that doesn't handle the savegame format correctly. --[[User:Wwylele|Wwylele]] ([[User talk:Wwylele|talk]]) 16:13, 2 December 2019 (CET))
* [[3DSExplorer]] supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).
* [[3DSExplorer]] supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).
* [https://github.com/wwylele/3ds-save-tool wwylele's 3ds-save-tool] supports extracting files from savegames and extdata. It properly reconstructs data from the DPFS tree and extracts files in directories hierarchy. It also contains a newer documentation of the save format but unfinished yet.
* [https://github.com/wwylele/3ds-save-tool wwylele's 3ds-save-tool] supports extracting files from savegames and extdata. It properly reconstructs data from the DPFS tree and extracts files in directories hierarchy.
** I will migrate the documentation here when I get time. Anyone is also welcome to do this before I do it. --[[User:Wwylele|Wwylele]] ([[User talk:Wwylele|talk]]) 13:18, 18 November 2017 (CET)
* [https://github.com/wwylele/3dsfuse-ex 3dsfuse-ex] similar to 3dsfuse, but supports savegame inner FS, proper DPFS handling, and automatic CMAC update. Still WIP.


[[セーブデータ|Japanese]]
[[セーブデータ|Japanese]]
Retrieved from "https://www.3dbrew.org/wiki/Savegames"