3dbrew

Savegames: Difference between revisions

← Older editNewer edit →
Wwylele (talk | contribs)
242 edits
→Tools: Add a new fuse tool. sorry for promoting my own tool, but I really think it is a bad idea to only keep an old broken (sorry again!) tool there
Wwylele (talk | contribs)
242 edits
Nuke the old content. Sorry for removing the next diagram, but it is really outdated. I will make a new one but please fix wiki image uploading first!
Line 1: Line 1:
This page describes the format and encryption of savegames contained in gamecards, SD/NAND, and SD/NAND [[extdata]]. You can find savegames from various 3DS games on the [[Games]] page.
This page describes the format and encryption of savegames contained in gamecards, SD and NAND. You can find savegames from various 3DS games on the [[Games]] page.


== Overview ==
Savegames are stored in [[DISA and DIFF|DISA container formats]](follow this link for the container format description). It forms a file system inside the inner content of the container. In this page only the inner file system format of the content is described.


=== Savegame Encryption ===
Unlike SD and NAND savegames, gamecard savegames has additional encryption + wear leveling layer. They are described in the following sections.


==== Repeating CTR Fail ====
== Gamecard savegame Encryption ===
 
=== Repeating CTR Fail ===
On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.
On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.


Line 11: Line 15:
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.


==== Savegame keyY ====
=== Savegame keyY ===


All gamecard and SD savegames are encrypted with AES-CTR. The base CTR for gamecard savegames is all-zero. The gamecard savegame [[AES|keyslots]]' keyY(these savegame keyslots use the hardware key-generator) is unique for each region and for each game. The [[NCSD]] partition flags determine the method used to generate this keyY. When the save [[NCSD]] flags checked by the running NATIVE_FIRM are all-zero, the system will use the repeating CTR, otherwise a proper CTR which never repeats within the image is used.
All gamecard and SD savegames are encrypted with AES-CTR. The base CTR for gamecard savegames is all-zero. The gamecard savegame [[AES|keyslots]]' keyY(these savegame keyslots use the hardware key-generator) is unique for each region and for each game. The [[NCSD]] partition flags determine the method used to generate this keyY. When the save [[NCSD]] flags checked by the running NATIVE_FIRM are all-zero, the system will use the repeating CTR, otherwise a proper CTR which never repeats within the image is used.
Line 37: Line 41:
|}
|}


===== [[2.0.0-2]] Hashed keyY and [[2.2.0-4]] Savegame Encryption =====
==== [[2.0.0-2]] Hashed keyY and [[2.2.0-4]] Savegame Encryption ====


When certain [[NCSD]] partition flags are set, a SHA-256 hash is calculated over the data from the CXI(same data used with the original plain keyY), and the 0x40-bytes read from a gamecard command(this 0x40-byte data is also read by [[Process_Services_PXI|GetRomId]], which is the gamecard-uniqueID). The first 0x10-bytes from this hash is used for the keyY. When flag[7] is set, the CTR will never repeat within the save image, unlike the original CTR-method. All games which had the retail NCSD image finalized after the [[2.2.0-4]] update(and contain [[2.2.0-4]]+ in the [[System Update CFA|System update partition]]), use this encryption method.
When certain [[NCSD]] partition flags are set, a SHA-256 hash is calculated over the data from the CXI(same data used with the original plain keyY), and the 0x40-bytes read from a gamecard command(this 0x40-byte data is also read by [[Process_Services_PXI|GetRomId]], which is the gamecard-uniqueID). The first 0x10-bytes from this hash is used for the keyY. When flag[7] is set, the CTR will never repeat within the save image, unlike the original CTR-method. All games which had the retail NCSD image finalized after the [[2.2.0-4]] update(and contain [[2.2.0-4]]+ in the [[System Update CFA|System update partition]]), use this encryption method.
Line 43: Line 47:
This keyY generation method was implemented with [[2.0.0-2]] via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until [[2.2.0-4]]. The hashed keyY flag[3] implemented with [[2.0.0-2]] was likely never used with retail gamecards.
This keyY generation method was implemented with [[2.0.0-2]] via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until [[2.2.0-4]]. The hashed keyY flag[3] implemented with [[2.0.0-2]] was likely never used with retail gamecards.


===== [[6.0.0-11]] Savegame keyY =====
==== [[6.0.0-11]] Savegame keyY ====


[[6.0.0-11]] implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new [[NCSD]] partition flags, all retail games which have the NCSD image finalized after the [[6.0.0-11]] release(and [[6.0.0-11]]+ in the system update partition) will have these flags set for using this new method.
[[6.0.0-11]] implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new [[NCSD]] partition flags, all retail games which have the NCSD image finalized after the [[6.0.0-11]] release(and [[6.0.0-11]]+ in the system update partition) will have these flags set for using this new method.
Line 53: Line 57:
This [[FIRM|Process9]] key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.
This [[FIRM|Process9]] key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.


=== Wear leveling ===
== Gamecard wear leveling ==


The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.
The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.
Line 97: Line 101:
* to calculate the checksum, a CRC16 of the block (with starting value 0xFFFF) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum
* to calculate the checksum, a CRC16 of the block (with starting value 0xFFFF) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum


=== AES CMAC header ===
== Partitions ==
 
{| class="wikitable"
|-
! Image offset
! Length
! Description
|-
| 0x00
| 0x10
| [[AES]]-CMAC over a 0x20-byte SHA256 hash(the keyslot used here uses the hardware key-scrambler).
|-
| 0x10
| 0xF0
| Zero padding
|}
 
This AES CMAC is used to "sign" the DISA/DIFF header. Each time the savegame is updated the hash stored in the DISA/DIFF is updated, therefore the CMAC must be updated each time the save is modified as well. SHA256_Update() is used to calculate the hash with the blocks described below.
 
==== Savegame Types ====
{| class="wikitable"
|-
! Type
! Description
|-
| CTR-EXT0
| SD/NAND [[Extdata]]
|-
| CTR-SYS0
| [[System SaveData]]
|-
| CTR-NOR0
| Gamecard Savegames
|-
| CTR-SAV0
| Savegames
|-
| CTR-SIGN
| SD Savegames
|-
| CTR-9DB0
| [[Title_Database|Title database]] extdata images
|}
 
==== Extdata SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
| 0x8
| u64 extdataID
|-
| 0x4
| 0 for Quota.dat, 1 otherwise
|-
| 0x8
| u64 imageID: First word is the hex ID from image filename, second word is the hex ID of the sub-dir under the <ExtdataIDLow> directory (all-zero for Quota.dat)
|-
| 0x100
| DIFF header
|}
 
==== System SaveData SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
| 0x8
| [[FS:OpenFile|SaveID]]
|-
| 0x100
| DISA header
|}
 
==== CTR-NOR0 SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
| 0x100
| DISA header
|}
 
==== CTR-SAV0 SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
|
| Input data, for gamecard savegames this is the output SHA-256 hash from CTR-NOR0.
|}
 
For gamecard savegames the output hash from this is used with the CMAC. This save-type is also used for SD savegames, for SD saves the input data is the 0x100-byte DISA header. For SD savegames, the calculated output hash is used with CTR-SIGN.
 
==== CTR-SIGN SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
| 0x8
| ProgramID/SaveID
|-
| 0x20
| SHA-256 hash from CTR-SAV0
|}
This is used for SD savegames, the calculated hash from this is used with the CMAC.
 
==== CTR-9DB0 SHA256 Blocks ====
{| class="wikitable"
|-
! Block Size
! Description
|-
| 0x8
| Savegame type
|-
| 0x4
| ID, each .db image has a separate ID.
|-
| 0x100
| DIFF header
|}
 
This is used for the SD/NAND /[[Title_Database|dbs]] .db extdata images, see [[Title_Database]] regarding AES-CMAC keyslots used here.
 
=== Partitions ===
 
There can be multiple partitions in the image.
The partitions are represented by tables of DIFI blobs inside a DISA/DIFF structure.
The order of the DIFI blobs is the order of the partitions in the image.
 
==== DISA ====
 
* This is located @ 0x100 in the image, following the CMAC header.
* If the uint32 @ 0x68 in the DISA(the low 8-bits) is non-zero, then the secondary table is is used, otherwise the primary table is used.
* If the table has more then 1 DIFI then the uint32 @ 0x168 is the offset from the DATA partition to the file base (masked with 0xFFFFFFFE).
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("DISA")
|-
| 0x04
| 4
| Magic Number (0x40000)
|-
| 0x08
| 8
| Total partition entries in a table
|-
| 0x10
| 8
| Offset to secondary partition table
|-
| 0x18
| 8
| Offset to primary partition table
|-
| 0x20
| 8
| Partition table size
|-
| 0x28
| 8
| SAVE Partition entry offset in the partition table
|-
| 0x30
| 8
| SAVE Partition entry length in the partition table
|-
| 0x38
| 8
| DATA Partition entry offset in the partition table
|-
| 0x40
| 8
| DATA Partition entry length in the partition table
|-
| 0x48
| 8
| SAVE Partition offset
|-
| 0x50
| 8
| SAVE Partition length
|-
| 0x58
| 8
| DATA Partition offset
|-
| 0x60
| 8
| DATA Partition length
|-
| 0x68
| 4
| Active table (and the offset to the filebase)
|-
| 0x6C
| 0x20
| Hash from active table
|-
| 0x8C
| 0x74
| Reserved
|}
 
* The hash in the DISA hashes the Active Table (starting from tables's offset to tables's offset + table length) with SHA256.
* The partition offsets are absolute offsets in the image.
* The SAVE partition offset is usually 0x1000. The SAVE/DATA partitions begins with the DPFS partitions, the relative offset for the IVFC partition data is specified by the DPFS header.
 
The DIFIs table at offset 0x200 in the image has 2 DIFIs when the DATA partition isn't used, 4 DIFIs otherwise. Each partition table contains the SAVE DIFI entry and optionally the DATA entry. The secondary partition table is located at offset 0x200 in the image, and the primary table follows the secondary table.
 
The non-active table is for backup.
 
==== DIFF ====
 
* This is the [[extdata]] equivalent of DISA, for extdata which use FS. DIFF is only used for extdata.
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("DIFF")
|-
| 0x04
| 4
| Magic Number (0x30000)
|-
| 0x08
| 8
| Secondary partition table offset
|-
| 0x10
| 8
| Primary partition table offset
|-
| 0x18
| 8
| Partition table length
|-
| 0x20
| 8
| File Base Offset
|-
| 0x28
| 8
| File Base Size
|-
| 0x30
| 4
| Active Partition Table (0 = Primary, 1 = Secondary)
|-
| 0x34
| 0x20
| Hash of the Active Partition Table
|-
| 0x54
| 0x8
| Unique Extdata Identifier
|-
| 0x5C
| 0xA4
| Reserved1
|}
 
* The Unique Extdata Identifier is a randomly generated series of bytes which is used to identify a specific extdata image. It's currently known to be used with the VSXE FST, if the ID doesn't match the one stored in the file's file entry, the extdata image isn't mounted in the FS.
 
==== DIFI ====
 
These 0x12C-byte blobs describe the partitions. Following each partition is an unused 0xFFFFFFFF cleartext word in the raw image. Every DIFI blob describes a partition. Partitions are catted together, so after the end of one partition is the beginning of the next.
 
For most games there's only 1 partition (The SAVE partition) and some (like Asphalt 3D, Steel Diver & Lego Star Wars III) has 2 partitions.
 
* 2 Partitions means that the files inside the SAVE partition is on the DATA partition.
* The DISA/DIFF headers support a maximum of 2 partitions.
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("DIFI")
|-
| 0x04
| 4
| Magic Number (0x10000)
|-
| 0x08
| 8
| Offset to "IVFC" blob in DIFI (Always 0x44)
|-
| 0x10
| 8
| Size of "IVFC" blob
|-
| 0x18
| 8
| Offset to "DPFS" blob in DIFI (Always 0xBC)
|-
| 0x20
| 8
| Size of "DPFS" blob
|-
| 0x28
| 8
| Offset to the hash in DIFI (Always 0x10C)
|-
| 0x30
| 8
| Size of this hash
|-
| 0x38
| 1
| When this byte is non-zero, this is a DATA partition, and IVFC level 4 is out side DPFS level 3, whose offset is specified by "IVFC level 4 offset"
|-
| 0x39
| 1
| DPFS level 1 selector. 0 = first one active; 1 = second one active
|-
| 0x3A
| 2
| Unknown
|-
| 0x3C
| 8
| IVFC level 4 offset (for DATA partitions)
|}
 
==== IVFC ====
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("IVFC")
|-
| 0x04
| 4
| Magic Number (0x20000)
|-
| 0x08
| 0x8
| Master hash size
|-
| 0x10
| 0x8
| Level 1 relative offset
|-
| 0x18
| 0x8
| Level 1 hashdata size
|-
| 0x20
| 0x4
| Level 1 block size, in log2
|-
| 0x24
| 0x4
| Reserved
|-
| 0x28
| 0x8
| Level 2 relative offset
|-
| 0x30
| 0x8
| Level 2 hashdata size
|-
| 0x38
| 0x4
| Level 2 block size, in log2.
|-
| 0x3C
| 0x4
| Reserved
|-
| 0x40
| 0x8
| Level 3 relative offset
|-
| 0x48
| 0x8
| Level 3 hashdata size
|-
| 0x50
| 0x4
| Level 3 block size, in log2.
|-
| 0x54
| 0x4
| Reserved
|-
| 0x58
| 8
| Level 4 filesystem relative offset
|-
| 0x60
| 8
| Level 4 filesystem size
|-
| 0x68
| 8
| Level 4 filesystem block size, in log2.
|-
| 0x70
| 8
| Unknown (usually 0x78=120)
|-
|}
 
* This savegame IVFC is almost identical to the [[RomFS]] IVFC, except for the additional filesystem level. Exactly like RomFS, each level except level4 is a hash-table where each hash entry hashes the data in the next level, padded to the log2 block size.
 
==== DPFS ====
 
{| class="wikitable"
|-
! Start
! Length
! Description
|-
| 0x00
| 4
| Magic ("DPFS")
|-
| 0x04
| 4
| Magic Number (0x10000)
|-
| 0x08
| 8
| Level 1 offset
|-
| 0x10
| 8
| Level 1 size
|-
| 0x18
| 8
| Level 1 block size, in log2 (unused?)
|-
| 0x20
| 8
| Level 2 offset
|-
| 0x28
| 8
| Level 2 size
|-
| 0x30
| 8
| Level 2 block size, in log2
|-
| 0x38
| 8
| Level 3 (IVFC partition) offset offset
|-
| 0x40
| 8
| Level 3 (IVFC partition) offset size
|-
| 0x48
| 8
| Level 3 (IVFC partition) block size, in log2
|-
|}
 
* Every level comes in pairs (concatenated), so the actual size of each level is twice as large as recorded in the "size" field. Level n (n = 1 or 2) is a bit-array (encoded in u32 array, with MSB as the first bit of each 32 bits), in which each bit corresponds to a block in level (n+1), indicating in which of the pair the block is the active one (0 means the first one, 1 means the second one). In this way, the actual valid data is scattered between the two IVFC partition images (Except for DATA partition, because its IVFC level 4 is out side DPFS level 3 and thus is not duplicated).
* The valid one in level 1 pair is chosen by "DPFS level 1 selector" in DIFI.
* The offsets contained in the DPFS are relative to the partition offset in the DISA/DIFF. The offsets from the IVFC are related to the IVFC partition (DPFS Level 3), after assembling the valid data.
 
 
'''DIFI Hash'''
 
The last 0x20-bytes of the partition following the DIFI, IVFC and DPFS is a SHA256 hash. The offset to this hash is stored in the DIFI. This hashes the IVFC level 1, with the buffer which is hashed aligned to the IVFC level 1 log2 block-size.
 
'''Summary Drawing'''
 
[[File:Sfimg_drawing.png]]


==== The SAVE partition ====
A savegame can have one or two partitions (specified by the [[DISA and DIFF|DISA container format]]). The first partition is called the SAVE partition, and the second optional partition is called the DATA partition.


* The SAVE filesystem works with a backup. There are two SAVE blocks inside the partition concatenated. Which SAVE block is the updated one is unknown yet.. (I'm guessing from experience that (image[0x100B] & 0x20) == 0x20 --> 1st SAVE --[[User:Elisherer|Elisherer]] 01:30, 18 October 2011 (CEST))
=== The SAVE partition ===
* See above about valid data selection in DPFS. The updated data is scattered between the two blocks. [[User:Wwylele|Wwylele]] ([[User talk:Wwylele|talk]]) 23:18, 14 November 2017 (CET)


'''Finding the folders table:'''
'''Finding the folders table:'''
Line 812: Line 301:
[[File:Sfsave_drawing.png]]
[[File:Sfsave_drawing.png]]


=== Initialization ===
== Initialization ==


When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.
When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.
Line 818: Line 307:
I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)


=== Fun Facts ===
== Fun Facts ==


If you have facts that you found out by looking at the binary files please share them here:
If you have facts that you found out by looking at the binary files please share them here:
Line 824: Line 313:
* From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)
* From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --[[User:Elisherer|Elisherer]] 22:41, 15 October 2011 (CEST)


=== Tools ===
== Tools ==


* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard.
* [https://github.com/3dshax/3ds/tree/master/3dsfuse 3dsfuse] supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard.
Retrieved from "https://www.3dbrew.org/wiki/Savegames"