3dbrew

Extdata: Difference between revisions

← Older edit
Wwylele (talk | contribs)
242 edits
→Tools: old extdata_tool is a dead link; Add new tools
Mt56 (talk | contribs)
2 edits
→SD Extdata: Add Swapnote JPN ExtdataID
 
(13 intermediate revisions by 7 users not shown)
Line 29: Line 29:
* Device file with <code>SubDirID = SubFileID = 00000000</code> doesn't exist. Other ID combinations can exists.
* Device file with <code>SubDirID = SubFileID = 00000000</code> doesn't exist. Other ID combinations can exists.
* Device file with <code>SubDirID = 00000000</code> and <code>SubFileID = 00000001</code> is the VSXE metadata file and must exist.
* Device file with <code>SubDirID = 00000000</code> and <code>SubFileID = 00000001</code> is the VSXE metadata file and must exist.
* Other files, besides <code>Quota.dat</code> and <code>00000000/00000001</code>, are normal sub files, are these device files one-to-one correspond to virtual files.
* Other files, besides <code>Quota.dat</code> and <code>00000000/00000001</code>, are normal sub files, are these device files one-to-one correspond to virtual files. They contain raw virtual file data in the DIFF inner content.
* <code>SubDirID = 00000000</code> is usually the only one device directory that can be seen. See [[#Device Directory Capacity]] for more information.
* <code>SubDirID = 00000000</code> is usually the only one device directory that can be seen. See [[#Device Directory Capacity]] for more information.


== Quota File ==
== Quota File ==


The inner data of <code>Quota.dat</code> is 0x48 bytes with the following format. The exact function of this file is unclear.
The inner data of <code>Quota.dat</code> is 0x48 bytes with the following format. The file seems to limit the extdata total size.


{| class="wikitable" border="1"
{| class="wikitable" border="1"
Line 51: Line 51:
| 0x08
| 0x08
| 4
| 4
| 0x1000, block size?
| 0x1000, block size
|-
|-
| 0x0C
| 0x0C
Line 57: Line 57:
| Always 126. Probably device directory capacity. See the [[#Device Directory Capacity]] more information.
| Always 126. Probably device directory capacity. See the [[#Device Directory Capacity]] more information.
|-
|-
| ...
| 0x10
|
 
| The meaning of other fields is unknown
|}
 
== Device Directory Capacity ==
 
A device directory in an extdata (a <code>&lt;SubDirID&gt;</code> directory) seems to have a maximum number of device files it can contain. For SD extdata, this maximum number seems to be hard-coded as 126. For NAND extdata, the number is probably indicated by a field in Quota.dat, which is, again, always 126 as observed. 3DS FS tries to put all device files in the device directory <code>00000000</code> if possible, and only when more than 126 files needed to add, a second device directory <code>00000001</code> and so on are created. However, few extdata have such amount of files to store, so the behavior lacks of use cases to confirm.
 
The number 126 is probably from some kind of other capacity of 128 with <code>&quot;.&quot;</code> and <code>&quot;..&quot;</code> entries reserved. It is theorized that this is to keep a FAT directory table, with 0x20 bytes for each entry, in one 0x1000 cluster. The motivation is unclear.
 
== VSXE File System Metadata ==
 
The inner data of <code>00000000/00000001</code> consists of the following components
 
* VSXE header
* Directory Hash Table
* File Hash Table
* File Allocation Table
* Data region
** Directory Entry Table
** File Entry Table
 
=== VSXE Header ===
 
{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| 4
| Magic &quot;VSXE&quot;
| Always 0?
|-
|-
| 0x04
| 0x14
| 4
| 4
| Magic 0x30000
| Max number of blocks
|-
| 0x08
| 8
| File system Information offset (0x138)
|-
| 0x10
| 8
| Image size in blocks
|-
|-
| 0x18
| 0x18
| 4
| 4
| Image block size
| Always 0?
|-
|-
| 0x1C
| 0x1C
| 4
| 4
| Padding
| Free blocks remained
|-
|-
| 0x20
| 0x20
| 8
| 4
| Unknown
| Always 0?
|-
| 0x24
| 4
| Always 0?
|-
|-
| 0x28
| 0x28
| 4
| 4
| 'Action' made on most recently mounted Extdata image
| Free blocks remained + (blocks occupied by the recently mounted file, specified by the ID below (0 if recently deleted))
|-
|-
| 0x2C
| 0x2C
| 4
| 4
| Unknown
| Always 0?
|-
|-
| 0x30
| 0x30
| 4
| 4
| D of most recently mounted Extdata image
| ID of most recently mounted file. Same as the one in [[Inner_FAT#Filesystem Header]]
|-
|-
| 0x34
| 0x34
| 4
| 4
| Unknown
| Always 0?
|-
|-
| 0x38
| 0x38
| 0x100
| Mount path, from most recently mounted Extdata image
|-
|
|
| Below is File system Information, which is assumed following the same layout as [[Savegames#SAVE Header
|-
| 0x138
| 4
| Unknown
|-
| 0x13C
| 4
| Data region block size
|-
| 0x140
| 8
| Directory hash table offset
|-
| 0x148
| 4
| Directory hash table bucket count
|-
| 0x14C
| 4
| Padding
|-
| 0x150
| 8
| File hash table offset
|-
| 0x158
| 4
| File hash table bucket count
|-
| 0x15C
| 4
| Padding
|-
| 0x160
| 8
| File allocation table offset
|-
| 0x168
| 4
| 4
| File allocation table entry count
| Always 0?
|-
|-
| 0x16C
| 0x3C
| 4
| 4
| Padding
| Always 0?
|-
|-
| 0x170
| 0x40
| 8
| Data region offset
|-
| 0x178
| 4
| 4
| Data region block count (= File allocation table entry count)
| Size in bytes of most recently mounted file (device file size). 0 if recently deleted
|-
|-
| 0x17C
| 0x44
| 4
| 4
| Padding
| Always 0?
|-
| 0x180
| 4
| Directory entry table starting block
|-
| 0x184
| 4
| Directory entry table block count
|-
| 0x188
| 4
| Maximum directory count
|-
| 0x18C
| 4
| Padding
|-
| 0x190
| 4
| File entry table starting block
|-
| 0x194
| 4
| File entry table block count
|-
| 0x198
| 4
| Maximum file count
|-
| 0x19C
| 4
| Padding
|}
|}


* All &quot;offsets&quot; are relative to the beginning of VSXE image. All &quot;starting block index&quot; are relative to the beginning of data region.
== Device Directory Capacity ==


=== File Allocation Table &amp; Data Region ===
A device directory in an extdata (a <code>&lt;SubDirID&gt;</code> directory) seems to have a maximum number of device files it can contain. For SD extdata, this maximum number seems to be hard-coded as 126. For NAND extdata, the number is probably indicated by a field in Quota.dat, which is, again, always 126 as observed. 3DS FS tries to put all device files in the device directory <code>00000000</code> if possible, and only when more than 126 files needed to add, a second device directory <code>00000001</code> and so on are created. However, few extdata have such amount of files to store, so the behavior lacks of use cases to confirm.


These function in the same way as [[Savegames#File Allocation Table|the file allocation in savegames]]. However, the only two &quot;files&quot; allocated in the data region are the directory entry table and file entry table, so the data region is usually pretty small, and the file allocation table is unchanged once created and has no free blocks. Thus, the offset and size of directory / file entry table can be found directly by <code>offset = entry_table_starting block * data_region_block_size + data_region_offset</code> and <code>size = entry_table_block_count * data_region_block_size</code>
The number 126 is probably from some kind of other capacity of 128 with <code>&quot;.&quot;</code> and <code>&quot;..&quot;</code> entries reserved. It is theorized that this is to keep a FAT directory table, with 0x20 bytes for each entry, in one 0x1000 cluster. The motivation is unclear.
 
=== Directory Hash Table &amp; File Hash Table ===


These function in the same way as [[Savegames#Directory Hash Table &amp; File Hash Table|those in savegames]]
== VSXE Filesystem ==


=== Directory Entry Table ===
This is one variant of the [[Inner FAT|FAT filesystem]]. Please refer to its page for the description of the filesystem. In general, device file <code>00000000/00000001</code> contains the metadata of the filesystem, while other device files (except for the Quota file) contains normal sub-files
 
This functions in the same way as [[Savegames#Directory Entry Table|the one in savegames]]. It lists all virtual directories in this extdata.
 
=== File Entry Table ===
 
This functions in a similar way as [[Savegames#File Entry Table|the one in savegames]]. It lists all virtual files in this extdata. However, the format of a (non-dummy) file entry is a little bit modified:
 
{| class="wikitable" border="1"
! Offset
! Length
! Description
|-
| 0x00
| 4
| Parent directory index in directory entry table
|-
| 0x04
| 16
| ASCII file name
|-
| 0x14
| 4
| Next sibling file index. 0 if this is the last one
|-
| 0x18
| 4
| Padding
|-
| 0x1C
| 4
| <s>First block index in data region</s> '''Always 0x80000000 because unused'''
|-
| 0x20
| 8
| <s>File size</s> '''Unique identifier'''
|-
| 0x28
| 4
| Padding?
|-
| 0x2C
| 4
| Index of the next file in the same hash table bucket. 0 if this is the last one
|}


Each non-dummy file entry corresponds to a device file. The path to the device file is generated by the following computation:
Each non-dummy file entry corresponds to a device file. The path to the device file is generated by the following computation:
Line 303: Line 140:
char extdata_path[...]; // &quot;.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;&quot;
char extdata_path[...]; // &quot;.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;&quot;
char device_path[...]; // output path
char device_path[...]; // output path
sprintfdevice_path, &quot;%s/%08x/%08x&quot;, extdata_path, SubDirID, SubFileID);
sprintf(device_path, &quot;%s/%08x/%08x&quot;, extdata_path, SubDirID, SubFileID);
</pre>
</pre>
When mounting extdata, the unique identifier is used to match the ID stored in subfile's [[DISA and DIFF#DIFF header|DIFF header]]. If the ID doesn't match, mounting will fail.
When mounting extdata, the unique identifier is used to match the ID stored in subfile's [[DISA and DIFF#DIFF header|DIFF header]]. If the ID doesn't match, mounting will fail.
Line 435: Line 272:
|  
|  
|-
|-
| ?
| 00000516
| 00000517
| 00000517
| 00000518
| 00000518
Line 539: Line 376:
|-
|-
| ?
| ?
| ?
| 00001132
| 00001131
| 00001131
| Fantasy Life
| Fantasy Life
Line 551: Line 388:
|-
|-
| ?
| ?
| ?
| 000012c8
| 000012ca
| 000012ca
| Mario vs. Donkey Kong: Tipping Stars
| Mario vs. Donkey Kong: Tipping Stars
Line 598: Line 435:
|  
|  
|-
|-
| ?
| 000016C6
| ?
| ?
| 00001678
| 00001678
Line 611: Line 448:
|-
|-
| ?
| ?
| ?
| 0000198e
| 0000198f
| 0000198f
| Animal Crossing: New Leaf - Welcome amiibo
| Animal Crossing: New Leaf - Welcome amiibo
Line 622: Line 459:
|  
|  
|-
|-
| ?
| 00001a2c
| ?
| ?
| 00001a2e
| 00001a2e
Line 709: Line 546:
Empty space is filled with 0xC-long sequences of 00 00 ... 07
Empty space is filled with 0xC-long sequences of 00 00 ... 07


=== Tools ===
== Tools ==
 
* [https://github.com/wwylele/3ds-save-tool 3ds-save-tool] - Extract/verifies extdata
* [https://github.com/wwylele/3ds-save-tool 3ds-save-tool] - Extract/verifies extdata
Retrieved from "https://www.3dbrew.org/wiki/Extdata"