Changes

Jump to navigation Jump to search
3,860 bytes added ,  12:07, 7 January 2018
Rework
This page describes the format and encryption of extdata, "&quot;extra data" &quot; stored on [[SD_Filesystem|SD card]] and [[Flash_Filesystem|NAND]].At, at:* nand/data/<ID>/extdata/<ExtdataID-High>* sdmc/Nintendo 3DS/<ID0>/<ID1>/extdata/<ExtdataID-High>
(* <code>nand/data/&lt;ID&gt;/extdata/&lt;ExtdataID-High is always 00000000 for SD, and always 00048000 for NAND) Some titles can have Quota.dat stored in these directories. The directory-name for these directories is the &gt;</code>* <code>sdmc/Nintendo 3DS/&lt;ID0&gt;/&lt;ID1&gt;/extdata/&lt;ExtdataID-Low. Then there's a sub-directory 00000000, which contains the actual extdata. Size and number of files in this dir varies per title.NAND stores the shared extdata and is structured exactly the same way, see [[Flash Filesystem]].High&gt;</code>
Extdata image 00000001 contains a VSXE partition ExtdataID-High is always 00000000 for the FSTSD, and always 00048000 for NAND. Regular apps can only mount SD extdata using the actual file data same extdataID which is stored in the subsequent CXI exheader. Therefore, regular apps which have the exheader extdataID set to zero can't use extdata. This restriction doesn't apply for shared extdata with extdataID high bitmask 0x48000 stored on NAND. System apps with a certain access right can mount arbitrary extdata. All NAND extdata is shared extdata, while all SD extdata is normal extdata images.
Regular apps can only mount SD extdata using the same extdataID which All data in this page is stored in the [[NCCH#CXI|CXI]] exheaderlittle-endian. Therefore, regular apps which have the exheader extdataID set to zero can't use extdata. This restriction doesn't apply for shared extdata with extdataID high bitmask 0x48000 stored on NAND. System apps with a certain access right All &quot;unused / padding&quot; fields can mount arbitrary extdata.All NAND extdata is shared extdata, while all SD extdata is normal extdata. Thus, normal extdata doesn't exist on NAND, and shared extdata doesn't exist on SD. The extdataID high excluding that bitmask is always zero for shared extdatacontain uninitialized data unless otherwise specified.
=== Format ===
All extdata is stored in [[DISA To avoid confusion, the terms '''device directory / file''' and DIFF|DIFF container files]] (follow this link for the container format description). The format description below is for the inner content of '''virtual directory / file''' are used with the containers.following meanings:
=== Filesystem ===* '''Device directory / file''' are the real directory / file stored on SD / NAND that can be seen under path <code>nand/data/&lt;ID&gt;/extdata/</code> or <code>sdmc/Nintendo 3DS/&lt;ID0&gt;/&lt;ID1&gt;/extdata/</code>.* '''Virtual directory / file''' are directory / file stored inside extdata virtual file system, which can be seen by applications in the mounted extdata archives.
Title An extdata contains a series consists of extdata images several device directories and files, which comprise an independent forms a file system. The first image (00000001) contains the VSXE (FST) partition, and subsequent images each containing a single file. Other extdata images, such as Quota.dat consisting of multiple virtual directories and [[Title Database|database extdata]], exist independent of a FSfiles.
An extdata with ID <code>ExtdataId</code> has the following device files: * <code>.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;/Quota.dat</code> (optional)* <code>.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;/&lt;SubDirID&gt;/&lt;SubFileID&gt;</code> Note: * All device files are [[DISA and DIFF|DIFF containers]]. '''All format description below is about the inner content of the containers'''. Please unwrap these files first according to the DIFF format description before reading them using the extdata format description below.* <code>Quota.dat</code> is only observed existing for NAND shared extdata.* <code>&lt;SubDirID&gt;</code> and <code>&lt;SubFileID&gt;</code> are 8-digit hex strings.* Device file with <code>SubDirID =SubFileID =00000000</code> doesn't exist. Other ID combinations can exists.* Device file with <code>SubDirID =00000000</code> and <code>SubFileID = 00000001</code> is the VSXE metadata file and must exist.* Other files, besides <code>Quota.dat</code> and <code>00000000/00000001</code>, are normal sub files, are these device files one-to-one correspond to virtual files.* <code>SubDirID = 00000000</code> is usually the only one device directory that can be seen. See [[#Device Directory Capacity]] for more information. ==Quota File == The inner data of <code>Quota.dat</code> is 0x48 bytes with the following format. The exact function of this file is unclear. {| class="wikitable"border="1"! Offset! Length! Description|-| 0x00| 4| Magic &quot;QUOT&quot;|-| 0x04| 4| Magic 0x30000|-| 0x08| 4| 0x1000, block size?|-| 0x0C| 4| Always 126. Probably device directory capacity. See the [[#Device Directory Capacity]] more information.
|-
| ...| | The meaning of other fields is unknown|} == Device Directory Capacity == A device directory in an extdata (a <code>&lt;SubDirID&gt;</code> directory) seems to have a maximum number of device files it can contain. For SD extdata, this maximum number seems to be hard-coded as 126. For NAND extdata, the number is probably indicated by a field in Quota.dat, which is, again, always 126 as observed. 3DS FS tries to put all device files in the device directory <code>00000000</code> if possible, and only when more than 126 files needed to add, a second device directory <code>00000001</code> and so on are created. However, few extdata have such amount of files to store, so the behavior lacks of use cases to confirm. The number 126 is probably from some kind of other capacity of 128 with <code>&quot;.&quot;</code> and <code>&quot;..&quot;</code> entries reserved. It is theorized that this is to keep a FAT directory table, with 0x20 bytes for each entry, in one 0x1000 cluster. The motivation is unclear. == VSXE File System Metadata == The inner data of <code>00000000/00000001</code> consists of the following components * VSXE header* Directory Hash Table* File Hash Table* File Allocation Table* Data region** Directory Entry Table** File Entry Table === VSXE Header === {| class="wikitable" border="1"! StartOffset
! Length
! Description
|-
| 0x00x00
| 4
| Database Magic ("&quot;VSXE")&quot;
|-
| 0x40x04
| 4
| Magic Number (0x30000)
|-
| 0x80x08
| 8
| Data Table OffsetFile system Information offset (0x138)
|-
| 0x10
| 8
| File Size, divided by the value at 0x18Image size in blocks
|-
| 0x18
| 84| Usually 0x1000Image block size|-| 0x1C| 4| Padding
|-
| 0x20
| 0x30
| 4
| ID D of most recently mounted Extdata image
|-
| 0x34
| 0x100
| Mount path, from most recently mounted Extdata image
|}-|
Data Table:|
{| class="wikitable"Below is File system Information, which is assumed following the same layout as [[Savegames#SAVE Header
|-
! Start| 0x138! Length| 4! Description| Unknown
|-
| 0x00x13C| 0x384| UnknownData region block size
|-
| 0x380x140
| 8
| Folder Table OffsetDirectory hash table offset|}-| 0x148| 4| Directory hash table bucket count===== Folder Table =====|-| 0x14CHeader:| 4{| class="wikitable"Padding
|-
! Start| 0x150! Length| 8! Description| File hash table offset
|-
| 0x00x158| 0x44| Equivalent to the Used Folder Entries + 1File hash table bucket count
|-
| 0x40x15C
| 4
| Equivalent to the Maximum Folder Entries + 1Padding|-| 0x160| 8| File allocation table offset
|-
| 0x80x168| 0x204| Unused|} Folder Entry:{| class="wikitable"File allocation table entry count
|-
! Start| 0x16C! Length| 4! Description| Padding
|-
| 0x00x170| 0x48| Parent Folder IndexData region offset
|-
| 0x40x178| 0x104| Folder Name Data region block count (ASCII= File allocation table entry count)
|-
| 0x140x17C| 0x44| Previous Folder's IndexPadding
|-
| 0x180x180| 0x44| Last Folder Index EntryDirectory entry table starting block
|-
| 0x1C0x184| 0x44| Last File Index EntryDirectory entry table block count
|-
| 0x200x188| 0x44| UnknownMaximum directory count
|-
| 0x2C0x18C| 0x44| Unknown|}* The folder id/index for the current entry is related to it's position in the Folder table. The folder table is accessed like an array of 0x28 byte chunks, with the header consuming index = 0, root directory at index = 1, and the subsequent folder entries following. ===== File Table ===== The location of the File table is calculated by aligning the end offset of the folder table to 0x1000 bytes. Header:{| class="wikitable"Padding
|-
! Start| 0x190! Length| 4! Description| File entry table starting block
|-
| 0x00x194| 0x44| Equivalent to the Used File Entries + 1entry table block count
|-
| 0x40x198
| 4
| Equivalent to the Maximum File Entries + 1file count
|-
| 0x80x19C| 0x284| UnusedPadding
|}
Folder * All &quot;offsets&quot; are relative to the beginning of VSXE image. All &quot;starting block index&quot; are relative to the beginning of data region. === File Allocation Table &amp; Data Region === These function in the same way as [[Savegames#File Allocation Table|the file allocation in savegames]]. However, the only two &quot;files&quot; allocated in the data region are the directory entry table and file entry table, so the data region is usually pretty small, and the file allocation table is unchanged once created and has no free blocks. Thus, the offset and size of directory / file entry table can be found directly by <code>offset = entry_table_starting block * data_region_block_size + data_region_offset</code> and <code>size = entry_table_block_count * data_region_block_size</code> === Directory Hash Table &amp; File Hash Table === These function in the same way as [[Savegames#Directory Hash Table &amp; File Hash Table|those in savegames]] === Directory EntryTable === This functions in the same way as [[Savegames#Directory Entry Table|the one in savegames]]. It lists all virtual directories in this extdata. === File Entry Table === This functions in a similar way as [[Savegames#File Entry Table|the one in savegames]]. It lists all virtual files in this extdata. However, the format of a (non-dummy) file entry is a little bit modified: {| class="wikitable"|-border="1"! StartOffset
! Length
! Description
|-
| 0x00x00| 0x44| Parent Folder Indexdirectory index in directory entry table
|-
| 0x40x04| 0x1016| File Name (ASCII)file name
|-
| 0x14
| 0x44| Previous File's IndexNext sibling file index. 0 if this is the last one
|-
| 0x18
| 0x44| UnknownPadding
|-
| 0x1C
| 0x44| Unknown<s>First block index in data region</s> '''Always 0x80000000 because unused'''
|-
| 0x20
| 0x88| <s>File size</s> '''Unique Extdata IDidentifier'''
|-
| 0x28
| 0x44| UnknownPadding?
|-
| 0x2C
| 0x44| UnknownIndex of the next file in the same hash table bucket. 0 if this is the last one
|}
* The file id/index for the current entry is related to it's position in the File Table, much like the folder entries in the Folder Table. The file table is accessed like an array of 0x30 byte chunks, with the header consuming index = 0, and the subsequent file entries following. The relationship between the index value of the file entry, and the actual file name of the extdata image that contains it it = index+1. For instance icon (the only file in every extdata), comes right after the header, with an index value of '1', and the icon is stored in extdata image '00000002'.
 
* The Unique Extdata ID, is the same value found in the DIFF header of the referenced extdata image for that file. The value changes most times the file in question is modified. When mounting an extdata image in the VSXE filesystem, if the file's extdata image doesn't have the expected Unique Extdata ID, it won't be mounted.
==== VSXE Filesystem structure ====Each non-dummy file entry corresponds to a device file. The path to the device file is generated by the following computation:
When extdata is created, these are *always* created regardless of whether the title actually uses them.<pre>// See previous section about this capacityconst uint32_t device_dir_capacity = 126;
* /icon This file contains / entry index is the extdata [[SMDH|icon]] displayed index in data management. This icon can only be written to by titles when creating extdatathe file entry table, titles would have to recreate extdata to change with the icon. This file can't be read directlyfirst dummy entry as// index = 0, instead it which is read via [[FS:ReadExtSaveDataIcon]]never used for a real file.* /user/ Contains file_index = 1 is reserved for the title's actual extdata VSXE Filesystem Metadata itself, so real files.* /boss/ Can contain [[SpotPass]] content. SpotPass content can only be downloaded to this /boss directorystarted from file_index = 2.uint32_t file_index = entry_index + 1;
User extdata and SpotPass extdata use separate [[FS:OpenArchive|mount]] points at uint32_t SubDirID = file_index /user and /boss. Therefore one mount can't access the other directory, and also can't access /icon.(The title's SpotPass extdata can be mounted by the title itself, if it uses SpotPass)device_dir_capacity;uint32_t SubFileID = file_index % pdevice_dir_capacity;
char extdata_path[...]; // &quot;.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;&quot;
char device_path[...]; // output path
sprintfdevice_path, &quot;%s/%08x/%08x&quot;, extdata_path, SubDirID, SubFileID);
</pre>
When mounting extdata, the unique identifier is used to match the ID stored in subfile's [[DISA and DIFF#DIFF header|DIFF header]]. If the ID doesn't match, mounting will fail.
Other optional but notable directories include:* /user/ExBanner This directory can optionally store [[Extended_Banner| extended banners]]. When this is available, this banner is displayed instead of the [[CXI]] ExeFS banner. COMMON.bin stores the common exbanner, while <regionlang_code>.bin stores an optional separate region/language specific banner.(regionlang_code can be "JPN_JP", "USA_EN", etc)== Virtual File System Structure ==
=== Extdata without an independent FS ===When extdata is created, these are ''always'' created regardless of whether the title actually uses them.
==== Quota* <code>/icon</code> This virtual file contains the extdata icon displayed in data management. This icon can only be written to by titles when creating extdata, titles would have to recreate extdata to change the icon. This file can't be read directly, instead it is read via FS:ReadExtSaveDataIcon.* <code>/user/</code> This virtual directory contains the title's actual extdata files.* <code>/boss/</code> This virtual directory can contain SpotPass content. SpotPass content can only be downloaded to this <code>/boss</code> virtual directory.dat ====
* This is contained in User extdata and SpotPass extdata use separate mount points at <code>/user</code> and <code>/boss</code>. Therefore one mount can't access the Quotaother virtual directory, and also can't access <code>/icon</code>.dat (The title's SpotPass extdata image.can be mounted by the title itself, if it uses SpotPass)
{| class="wikitable"|-! Start! Length! Description|-| 0x0| 4| Magic ("QUOT")|-| 0x4| 4| Magic Number (0x30000)|-| 0x8| 8| Unknown|-| 0x10| 0x38| Unknown|}It's unknown what this is used for. ==== Database Extdata ====Other optional but notable directories include:
See [[Title Database|here]]* <code>/user/ExBanner</code> This virtual directory can optionally store extended banners.When this is available, this banner is displayed instead of the CXI ExeFS banner. <code>COMMON.bin</code> stores the common exbanner, while <code>&lt;regionlang_code&gt;.bin</code> stores an optional separate region/language specific banner.(regionlang_code can be &quot;JPN_JP&quot;, &quot;USA_EN&quot;, etc)
=== SD Extdata ===
Usually the ExtdataID low is in the format '00<Unique ID>'
|}
=== NAND Shared Extdata ===
{| class="wikitable" border="1"
|-
242

edits

Navigation menu