Changes

Jump to navigation Jump to search
12,317 bytes added ,  18:18, 1 January 2018
Start refactoring savegame/extdata documentation.
This is a page under construction. The purpose is to refactor some non-savegame specific stuff out of the [[Savegames]] page and correct mistakes.

== Overview ==

DISA and DIFF are two container formats. They are very similar and are used for various purpose in 3DS. Here is a summary table of their usage, the CMAC type and key slot used (the meaning of these is explained in the next section):

{| class="wikitable" border="1"
! Usage
! Media
! Container format
! CMAC type
! CMAC [[AES Registers|Keyslot]]
|-
| [[Savegames]]
| Gamecard
| DISA
| CTR-SAV0
| 0x19
|-
| [[Savegames]]
| [[SD Filesystem|SD]]
| DISA
| CTR-SIGN
| 0x30
|-
| [[Savegames|System SaveData]]
| [[Flash Filesystem|NAND]]
| DISA
| CTR-SYS0
| 0x30
|-
| [[Extdata|Private Extdata]]
| [[SD Filesystem|SD]]
| DIFF
| CTR-EXT0
| 0x30
|-
| [[Extdata|Shared Extdata]]
| [[Flash Filesystem|NAND]]
| DIFF
| CTR-EXT0
| 0x30
|-
| [[Title Database]]
| [[SD Filesystem|SD]]
| DIFF
| CTR-9DB0
| 0x30
|-
| [[Title Database]]
| [[Flash Filesystem|NAND]]
| DIFF
| CTR-9DB0
| 0x0B
|}

This page only describe the container formats. The inner format for each usage is described in the corresponding page.

== Encryption ==

DISA and DIFF formats don't have their own encryption specification. They follows the encryption method defined by their media:

* Gamecard savegames has special wear leveling + encryption layers. See [[Savegames]] for detail.
* Files on SD follows [[SD Filesystem|the general SD filesystem encryption rule]].
* Files on NAND are in cleartext, but encrypted by [[Flash Filesystem|the NAND partition encryption]].

== Format ==

A DISA / DIFF file consists of following components:

* 0x100-byte AES CMAC
* 0x100-byte Header
* Secondary partition table
** Contains one or two partition descriptors, depending on the number of partitions.
* Primary partition table
** Same layout as the secondary one
* Partition A
* Partition B (optional)
** can only exist for DISA.

== AES CMAC ==

== Header ==

The header defines the rest components of the file (partitions and their tables). All offsets in the header are relative to the beginning of the DISA/DIFF file, except for partition descriptor offsets, which are relative to the beginning of the (active) partition table. DISA and DIFF have different header format.

=== DISA header ===

{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Magic "DISA"
|-
| 0x04
| 4
| Magic 0x40000
|-
| 0x08
| 4
| Partition count, 1 or 2
|-
| 0x0C
| 4
| Padding
|-
| 0x10
| 8
| Secondary partition table offset
|-
| 0x18
| 8
| Primary partition table offset
|-
| 0x20
| 8
| Partition table size
|-
| 0x28
| 8
| Partition A descriptor offset in the partition table
|-
| 0x30
| 8
| Partition A descriptor size
|-
| 0x38
| 8
| Partition B descriptor offset in the partition table
|-
| 0x40
| 8
| Partition B descriptor size
|-
| 0x48
| 8
| Partition A offset
|-
| 0x50
| 8
| Partition A size
|-
| 0x58
| 8
| Partition B offset
|-
| 0x60
| 8
| Partition B size
|-
| 0x68
| 1
| Active partition table, 0 = primary, 1 = secondary
|-
| 0x69
| 3
| Padding
|-
| 0x6C
| 0x20
| SHA-256 over the active partition table
|-
| 0x8C
| 0x74
| Unused
|}

Note:

* When partition count is 1, there is no partition B and all of its related fields are zero.

=== DIFF header ===

{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Magic "DIFF"
|-
| 0x04
| 4
| Magic 0x30000
|-
| 0x08
| 8
| Secondary partition table/descriptor offset
|-
| 0x10
| 8
| Primary partition table/descriptor offset
|-
| 0x18
| 8
| Partition table/descriptor size
|-
| 0x20
| 8
| Partition (A) offset
|-
| 0x28
| 8
| Partition (A) size
|-
| 0x30
| 4
| Active partition descriptor, 0 = primary, 1 = secondary
|-
| 0x34
| 0x20
| SHA-256 over the active partition table/descriptor
|-
| 0x54
| 8
| Unique identifier
|-
| 0x5C
| 0xA4
| Unused, might contain leftover data
|}

Note:

* Since DIFF can only contain one partition, a partition table can only have one partition descriptor, so they become the same concept here.
* See [[Extdata]] for its usage of the unique identifier field. For title database files, this field is zero.

== Partition table & partition descriptor ==

Their are two partition tables, but only one of them is active. When operating on a DISA / DIFF file, 3DS FS alternately activate one of the two tables, presumably for data backup or atomic file writing. A newly created DISA / DIFF file may have invalid data in its the inactive partition table.

One partition table contains one or two partition descriptors , each of which describe the layout of one partition. A partition descriptor contains the following components:

* DIFI header
* IVFC descriptor
* DPFS descriptor
* Partition master hash

=== DIFI header ===

The DIFI header locates at the beginning of a partition descriptor. This header defines the rest components of the partition descriptor (IVFC descriptor, DPFS descriptor and partition hash). All offsets are relative to the beginning of the partition descriptor, except for <code>External IVFC level 4 offset</code>, which is related to the beginning of the partition.

{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Magic &quot;DIFI&quot;
|-
| 0x04
| 4
| Magic 0x10000
|-
| 0x08
| 8
| IVFC descriptor offset
|-
| 0x10
| 8
| IVFC descriptor size
|-
| 0x18
| 8
| DPFS descriptor offset
|-
| 0x20
| 8
| DPFS descriptor size
|-
| 0x28
| 8
| Partition hash offset
|-
| 0x30
| 8
| Partition hash size
|-
| 0x38
| 1
| If none zero, enable external IVFC level 4.
|-
| 0x39
| 1
| DPFS tree level 1 selector
|-
| 0x3A
| 2
| Padding
|-
| 0x3C
| 8
| External IVFC level 4 offset
|}

Note:

* The meaning of fields after 0x38 are explained in the section [[#Partition]]

=== IVFC descriptor ===

This header defines each level of IVFC tree (explained in the section [[#Partition]]). All the offsets are relative to the beginning of DPFS level 3.

{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Magic &quot;IVFC&quot;
|-
| 0x04
| 4
| Magic 0x20000
|-
| 0x08
| 8
| Master hash size = partition master hash size in DIFI header
|-
| 0x10
| 8
| IVFC level 1 offset
|-
| 0x18
| 8
| IVFC level 1 size
|-
| 0x20
| 4
| IVFC level 1 block size in log2
|-
| 0x24
| 4
| Padding
|-
| 0x28
| 8
| IVFC level 2 offset
|-
| 0x30
| 8
| IVFC level 2 size
|-
| 0x38
| 4
| IVFC level 2 block size in log2
|-
| 0x3C
| 4
| Padding
|-
| 0x40
| 8
| IVFC level 3 offset
|-
| 0x48
| 8
| IVFC level 3 size
|-
| 0x50
| 4
| IVFC level 3 block size in log2
|-
| 0x54
| 4
| Padding
|-
| 0x58
| 8
| IVFC level 4 offset (unused for external IVFC level 4?)
|-
| 0x60
| 8
| IVFC level 4 size
|-
| 0x68
| 8
| IVFC level 4 block size in log2
|-
| 0x70
| 8
| IVFC descriptor size? usually 0x78
|}

=== DPFS descriptor ===

This header defines each level of DPFS tree (explained in the section [[#Partition]]). All the offsets are relative to the beginning of the partition.

{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Magic &quot;DPFS&quot;
|-
| 0x04
| 4
| Magic 0x10000
|-
| 0x08
| 8
| DPFS level 1 offset
|-
| 0x10
| 8
| DPFS level 1 size
|-
| 0x18
| 4
| DPFS level 1 block size in log2 (unused?)
|-
| 0x1C
| 4
| Padding
|-
| 0x20
| 8
| DPFS level 2 offset
|-
| 0x28
| 8
| DPFS level 2 size
|-
| 0x30
| 4
| DPFS level 2 block size in log2
|-
| 0x34
| 4
| Padding
|-
| 0x38
| 8
| DPFS level 3 offset
|-
| 0x40
| 8
| DPFS level 3 size
|-
| 0x48
| 4
| DPFS level 3 block size in log2
|-
| 0x4C
| 4
| Padding
|}

=== Partition master hash ===

This is a SHA-256 hash list over IVFC level 1. See [[#IVFC tree]] for explanation.

== Partition ==

A partition can have two types of layout. This is determined by the field DIFF + 0x38 (<code>Enable external IVFC level 4</code>).

The layout type 0 (external IVFC level 4 disabled) contains

* DPFS level 1
* DPFS level 2
* DPFS level 3, and inside
** IVFC level 1
** IVFC level 2
** IVFC level 3
** IVFC level 4 (the actual content data)

The layout type 1 (external IVFC level 4 enabled) contains

* DPFS level 1
* DPFS level 2
* DPFS level 3, and inside
** IVFC level 1
** IVFC level 2
** IVFC level 3
* IVFC level 4 (the actual content data, note that this is out side DPFS level 3)

=== DPFS tree ===

Everything inside the DPFS tree comes in pairs, and at one time only one of a pair is active. The tree is probably designed for atomic writing: for a file writing operation, it writes to the inactive part, then commit the data by switching a bit to activate it.

Each level of DPFS tree consists of a pair of chunks. The size of one chunk is defined as it in the DPFS descriptor, so the total size of a level is actually twice as large as the size recorded in the descriptor. For level 1 and 2, each chunk is a bit array, in which each bit corresponds to a block in the next level (the block size of the next level is also defined in the DPFS descriptor). This bit indicates which one of the pair in the next level is active for this block: 0 means the first one and 1 means the second one. The active chunk of level 1 is selected by <code>DPFS tree level 1 selector</code> in the DIFI header. The bit array is encoded in u32 array, with MSB as the first bit of each 32 bits.

To access data in level 3, one need to check the bits in level 1 and level 2 to know which chunk of level 3 is active for the accessing location. For example, for a following configuration:

* Level 1: size = 4 bytes = 32 bits
* Level 2: size = 0x380 bytes = 0x1C00 bits, block size = 0x80 bytes
* Level 3: size = 0x1B7F000, block size = 0x1000, block size = 0x1000 bytes

and one want to read byte at 0x1234567 of level 3, the following calculation is performed:

* get level 2 bit index <code>0x1234567 / 0x1000 = 0x1234</code>, and its byte location <code>0x1234 / 8 = 0x246</code>
* get level 1 bit index <code>0x246 / 0x80 = 4</code>
* get <code>level1_selector</code> from DIFI header
* read <code>level2_selector = Level1[level1_selector].bits[4]</code>;
* read <code>level3_selector = Level2[level2_selector].bits[0x1234]</code>;
* read <code>data = Level3[level3_selector].bytes[0x1234567]</code> as the final data
* in the code above <code>Levelx[k]</code> means the <code>k</code>-th chunk in level <code>x</code>, where <code>k = 0, 1</code>. <code>.bits[n]</code> is expanded to <code>(.u32_array[n / 32] &gt;&gt; (31 - n % 32)) &amp; 1</code> as the bit array is encoded in u32 array.

Effectively, the active data is scattered among the two level 3 chunk. One can assemble the whole active level 3 image following the same rule.

=== IVFC tree ===

The IVFC tree is used for data verification. It is very similar to the IVFC tree in RomFS*, except that it has an additional level in DISA / DIFF. For level 1, 2 and 3, each level is a list of SHA-256 hash, of which each corresponds to a block of the next level, padded to block size (the block size of the next level is defined in the IVFC descriptor).

The partition master hash in the partition descriptor can be seen as IVFC level 0, which hashes level 1 following the same rule. The partition hash is usually 0x20 long, consisting only one hash. This is because most DISA / DIFF files are not large enough to have multiple hashes on the top level, which isn't the case for some title database files.

However, not all data are hashed - only ranges that have been written with valid data are properly hashed.

Level 4 is the actual content of the partition, which is what the container format essentially contains.

== Extract content from a DISA / DIFF container ==

* Find the active partition table and the partition(s).
* Unwrap DPFS tree of partition(s) by reconstructing active data.
* Unwrap IVFC tree. Either take out level 4 directly, or, better, verify all the hashes and poison the data that is not properly hashed.
* The IVFC level 4 is the inner content of the file. The format of it varies among different usage. Refer their own page for further extraction.

== Chain of Trust ==

* AES CMAC verifies the header.
* The header verifies the active partition table via the table hash.
* In the partition table, each descriptor verifies level 1 of its IVFC tree via the master hash.
* Each IVFC level verifies the next level, until the level 4, which is the inner content.
242

edits

Navigation menu