3dbrew

Changes

Extdata (edit)

Revision as of 20:55, 22 March 2024

14,400 bytes added ,  22 March
→‎Tools
Line 1: Line 1: −
This page describes the format and crypto of extdata, "extra data" and StreetPass data stored on [[SD_Filesystem|SD card]].
+
This page describes the format and encryption of extdata, "extra data" stored on SD card and NAND, at:
−
At /Nintendo 3DS/<SomeID>/<SomeID>/extdata/00000000 is directories for each title, with Quota.dat inside each dir. Then there's a sub-directory 00000000, which contains the actual extdata. Size and number of files in this dir varies per title.
     −
These files are encrypted with AES-CTR. The key is unknown, might be title and console unique. The base CTR is fixed: the CTR never changes after each write. Thus it is possible to obtain some cleartext by XORing one file with a newer file, where the newer file overwrote zeros in the original file with non-zero data. Some Ridge Racer extdata from some replay data: https://gist.github.com/d6b646c581d386b4fa83
+
* <code>nand/data/&lt;ID&gt;/extdata/&lt;ExtdataID-High&gt;</code>
 +
* <code>sdmc/Nintendo 3DS/&lt;ID0&gt;/&lt;ID1&gt;/extdata/&lt;ExtdataID-High&gt;</code>
 +
 
 +
ExtdataID-High is always 00000000 for SD, and always 00048000 for NAND. Regular apps can only mount SD extdata using the same extdataID which is stored in the CXI exheader. Therefore, regular apps which have the exheader extdataID set to zero can't use extdata. This restriction doesn't apply for shared extdata with extdataID high bitmask 0x48000 stored on NAND. System apps with a certain access right can mount arbitrary extdata. All NAND extdata is shared extdata, while all SD extdata is normal extdata.
 +
 
 +
All data in this page is little-endian. All &quot;unused / padding&quot; fields can contain uninitialized data unless otherwise specified.
 +
 
 +
= Format =
 +
 
 +
To avoid confusion, the terms '''device directory / file''' and '''virtual directory / file''' are used with the following meanings:
 +
 
 +
* '''Device directory / file''' are the real directory / file stored on SD / NAND that can be seen under path <code>nand/data/&lt;ID&gt;/extdata/</code> or <code>sdmc/Nintendo 3DS/&lt;ID0&gt;/&lt;ID1&gt;/extdata/</code>.
 +
* '''Virtual directory / file''' are directory / file stored inside extdata virtual file system, which can be seen by applications in the mounted extdata archives.
 +
 
 +
An extdata consists of several device directories and files, which forms a file system consisting of multiple virtual directories and files.
 +
 
 +
An extdata with ID <code>ExtdataId</code> has the following device files:
 +
 
 +
* <code>.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;/Quota.dat</code> (optional)
 +
* <code>.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;/&lt;SubDirID&gt;/&lt;SubFileID&gt;</code>
 +
 
 +
Note:
 +
 
 +
* All device files are [[DISA and DIFF|DIFF containers]]. '''All format description below is about the inner content of the containers'''. Please unwrap these files first according to the DIFF format description before reading them using the extdata format description below.
 +
* <code>Quota.dat</code> is only observed existing for NAND shared extdata.
 +
* <code>&lt;SubDirID&gt;</code> and <code>&lt;SubFileID&gt;</code> are 8-digit hex strings.
 +
* Device file with <code>SubDirID = SubFileID = 00000000</code> doesn't exist. Other ID combinations can exists.
 +
* Device file with <code>SubDirID = 00000000</code> and <code>SubFileID = 00000001</code> is the VSXE metadata file and must exist.
 +
* Other files, besides <code>Quota.dat</code> and <code>00000000/00000001</code>, are normal sub files, are these device files one-to-one correspond to virtual files. They contain raw virtual file data in the DIFF inner content.
 +
* <code>SubDirID = 00000000</code> is usually the only one device directory that can be seen. See [[#Device Directory Capacity]] for more information.
 +
 
 +
== Quota File ==
 +
 
 +
The inner data of <code>Quota.dat</code> is 0x48 bytes with the following format. The file seems to limit the extdata total size.
 +
 
 +
{| class="wikitable" border="1"
 +
! Offset
 +
! Length
 +
! Description
 +
|-
 +
| 0x00
 +
| 4
 +
| Magic &quot;QUOT&quot;
 +
|-
 +
| 0x04
 +
| 4
 +
| Magic 0x30000
 +
|-
 +
| 0x08
 +
| 4
 +
| 0x1000, block size
 +
|-
 +
| 0x0C
 +
| 4
 +
| Always 126. Probably device directory capacity. See the [[#Device Directory Capacity]] more information.
 +
|-
 +
| 0x10
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x14
 +
| 4
 +
| Max number of blocks
 +
|-
 +
| 0x18
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x1C
 +
| 4
 +
| Free blocks remained
 +
|-
 +
| 0x20
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x24
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x28
 +
| 4
 +
| Free blocks remained + (blocks occupied by the recently mounted file, specified by the ID below (0 if recently deleted))
 +
|-
 +
| 0x2C
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x30
 +
| 4
 +
| ID of most recently mounted file. Same as the one in [[Inner_FAT#Filesystem Header]]
 +
|-
 +
| 0x34
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x38
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x3C
 +
| 4
 +
| Always 0?
 +
|-
 +
| 0x40
 +
| 4
 +
| Size in bytes of most recently mounted file (device file size). 0 if recently deleted
 +
|-
 +
| 0x44
 +
| 4
 +
| Always 0?
 +
|}
 +
 
 +
== Device Directory Capacity ==
 +
 
 +
A device directory in an extdata (a <code>&lt;SubDirID&gt;</code> directory) seems to have a maximum number of device files it can contain. For SD extdata, this maximum number seems to be hard-coded as 126. For NAND extdata, the number is probably indicated by a field in Quota.dat, which is, again, always 126 as observed. 3DS FS tries to put all device files in the device directory <code>00000000</code> if possible, and only when more than 126 files needed to add, a second device directory <code>00000001</code> and so on are created. However, few extdata have such amount of files to store, so the behavior lacks of use cases to confirm.
 +
 
 +
The number 126 is probably from some kind of other capacity of 128 with <code>&quot;.&quot;</code> and <code>&quot;..&quot;</code> entries reserved. It is theorized that this is to keep a FAT directory table, with 0x20 bytes for each entry, in one 0x1000 cluster. The motivation is unclear.
 +
 
 +
== VSXE Filesystem ==
 +
 
 +
This is one variant of the [[Inner FAT|FAT filesystem]]. Please refer to its page for the description of the filesystem. In general, device file <code>00000000/00000001</code> contains the metadata of the filesystem, while other device files (except for the Quota file) contains normal sub-files
 +
 
 +
Each non-dummy file entry corresponds to a device file. The path to the device file is generated by the following computation:
 +
 
 +
<pre>// See previous section about this capacity
 +
const uint32_t device_dir_capacity = 126;
 +
 
 +
// entry index is the index in the file entry table, with the first dummy entry as
 +
// index = 0, which is never used for a real file.
 +
// file_index = 1 is reserved for the VSXE Filesystem Metadata itself, so real files
 +
// started from file_index = 2.
 +
uint32_t file_index = entry_index + 1;
 +
 
 +
uint32_t SubDirID = file_index / device_dir_capacity;
 +
uint32_t SubFileID = file_index % pdevice_dir_capacity;
 +
 
 +
char extdata_path[...]; // &quot;.../extdata/&lt;ExtdataID-High&gt;/&lt;ExtdataId-Low&gt;&quot;
 +
char device_path[...]; // output path
 +
sprintf(device_path, &quot;%s/%08x/%08x&quot;, extdata_path, SubDirID, SubFileID);
 +
</pre>
 +
When mounting extdata, the unique identifier is used to match the ID stored in subfile's [[DISA and DIFF#DIFF header|DIFF header]]. If the ID doesn't match, mounting will fail.
 +
 
 +
== Virtual File System Structure ==
 +
 
 +
When extdata is created, these are ''always'' created regardless of whether the title actually uses them.
 +
 
 +
* <code>/icon</code> This virtual file contains the extdata icon displayed in data management. This icon can only be written to by titles when creating extdata, titles would have to recreate extdata to change the icon. This file can't be read directly, instead it is read via FS:ReadExtSaveDataIcon.
 +
* <code>/user/</code> This virtual directory contains the title's actual extdata files.
 +
* <code>/boss/</code> This virtual directory can contain SpotPass content. SpotPass content can only be downloaded to this <code>/boss</code> virtual directory.
 +
 
 +
User extdata and SpotPass extdata use separate mount points at <code>/user</code> and <code>/boss</code>. Therefore one mount can't access the other virtual directory, and also can't access <code>/icon</code>.(The title's SpotPass extdata can be mounted by the title itself, if it uses SpotPass)
 +
 
 +
Other optional but notable directories include:
 +
 
 +
* <code>/user/ExBanner</code> This virtual directory can optionally store extended banners. When this is available, this banner is displayed instead of the CXI ExeFS banner. <code>COMMON.bin</code> stores the common exbanner, while <code>&lt;regionlang_code&gt;.bin</code> stores an optional separate region/language specific banner.(regionlang_code can be &quot;JPN_JP&quot;, &quot;USA_EN&quot;, etc)
 +
 
 +
== SD Extdata ==
 +
Usually the ExtdataID low is in the format '00<Unique ID>'
 +
 
 +
{| class="wikitable" border="1"
 +
|-
 +
! JPN ExtdataID
 +
! USA ExtdataID
 +
! EUR ExtdataID
 +
! Description
 +
! Extdata images
 +
|-
 +
| 00000082
 +
| 0000008f
 +
| 00000098
 +
| [[Home Menu]] extdata, this contains home-menu savedata and cached icons for applications.
 +
|
 +
|-
 +
| 00000200
 +
| 00000210
 +
| 00000220
 +
| [[System_Settings|System Settings]] extdata added with [[2.0.0-2]].
 +
|
 +
|-
 +
| 00000207
 +
| 00000217
 +
| 00000227
 +
| [[Mii Maker]], contains an ExBanner
 +
| [https://dl.dropbox.com/u/20520664/extdata00000217.zip cleartext]
 +
|-
 +
| 00000208
 +
| 00000218
 +
| 00000228
 +
| Streetpass Mii Plaza
 +
| 11 mb big!
 +
|-
 +
| 00000209
 +
| 00000219
 +
| 00000229
 +
| [[eShop]], contains store music in AAC format.
 +
|
 +
|-
 +
| 0000020b
 +
| 0000021b
 +
| 0000022b
 +
| Nintendo Zone
 +
|
 +
|-
 +
| 0000020d
 +
| 0000021d
 +
| 0000022d
 +
| Face Raiders, likely contains an ExBanner
 +
|
 +
|-
 +
| 000002cc
 +
| 000002cd
 +
| 000002ce
 +
| [[Home Menu]] theme
 +
|
 +
|-
 +
| ?
 +
| 000004aa
 +
| 000004ab
 +
| Nintendo Video Extra Data
 +
This is where the video files are stored, and [[Nintendo_Video|includes]] the thumbnail, the description, and possibly some checksum info in each video file stored in the extdata images. There are always 9 files within the subdirectory "00000000" of this folder, even without any videos downloaded. The files are "00000001" - "00000009", and "00000003" - "00000008" have the same filesize of 50.7 MB. It is possible to restore the older videos by overwriting all the files within this directory. Provided of course you have made a backup of the files before hand, by copying all the files within this directory to your computer. As far I'm aware its not possible to mix and match the files in order to get certain videos in one grouping, ie. having all 3 Zelda orchestral recordings in one group of 4 Nintendo videos.
 +
|
 +
|-
 +
|00000306
 +
|00000308
 +
|00000307
 +
|Mario Kart 7
 +
|
 +
|-
 +
| 0000030b
 +
| 0000030d
 +
| 0000030c
 +
| Nintendogs + Cats
 +
|
 +
|-
 +
| 00000326
 +
| 00000326
 +
| 00000326
 +
| Pokédex 3D
 +
|
 +
|-
 +
| 00000305
 +
| 0000032d
 +
| 0000033c
 +
| Super Street Fighter IV 3D
 +
|
 +
|-
 +
| 00000328
 +
| 00000358
 +
| 0000033b
 +
| Ridge Racer 3D
 +
|
 +
|-
 +
| ?
 +
| 0000034d
 +
| 00000402
 +
| Samurai Warriors Chronicles
 +
|
 +
|-
 +
| ?
 +
| 0000034f
 +
| 0000038a
 +
| Dead or Alive Dimensions
 +
|
 +
|-
 +
| 00000481
 +
| N/A
 +
| N/A
 +
| Monster Hunter Tri G (Download-Quests)
 +
|
 +
|-
 +
| ?
 +
| 00000517
 +
| 00000518
 +
| Swapnote
 +
|
 +
|-
 +
| 0000055d
 +
| 0000055d
 +
| 0000055d
 +
| Pokémon X<br>Pokémon Y
 +
|
 +
|-
 +
| ?
 +
| 00000725
 +
| 00000724
 +
| Ambassador Certificate
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 000007af
 +
| New Super Mario Bros. 2
 +
|
 +
|-
 +
| ?
 +
| 00000863
 +
| 00000864
 +
| Animal Crossing: New Leaf
 +
|
 +
|-
 +
| ?
 +
| 00000a85
 +
| 00000a86
 +
| Professor Layton and the Miracle Mask<br>Professor Layton and the Azran Legacy
 +
German Version ExtdataID is 00000a87
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00000b4f
 +
| Fullblox / Crashmo
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00000ba9
 +
| Pokémon Mystery Dungeon: Gates to Infinity
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00000c24
 +
| Denpa men
 +
|
 +
|-
 +
| 00000c73
 +
| 00000c73
 +
| 00000c73
 +
| Save Data Transfer Tool
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00000d9a
 +
| Donkey Kong Country™<br>Returns 3D: Trailer
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00000ea6
 +
| Etrian Odyssey IV
 +
|
 +
|-
 +
| ?
 +
| 00000edf
 +
| 00000ee0
 +
| Super Smash Bros. for Nintendo 3DS
 +
|
 +
|-
 +
| ?
 +
| 00000f14
 +
| 00000f1e
 +
| Phoenix Wright: Ace Attorney - Dual Destinies
 +
|
 +
|-
 +
| ?
 +
| 00001007
 +
| 00001005
 +
| Professor Layton vs Phoenix Wright: Ace Attorney
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001062
 +
| Nintendo Pocket Football Club
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 0000111c
 +
| Yoshi's New Island
 +
|
 +
|-
 +
| ?
 +
| 00001132
 +
| 00001131
 +
| Fantasy Life
 +
|
 +
|-
 +
| 000011c5
 +
| 000011c5
 +
| 000011c5
 +
| Pokémon Omega Ruby<br>Pokémon Alpha Sapphire
 +
|
 +
|-
 +
| ?
 +
| 000012c8
 +
| 000012ca
 +
| Mario vs. Donkey Kong: Tipping Stars
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001499
 +
| Korg DSN-12
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 000014f2
 +
| Animal Crossing: Happy Home Designer
 +
|
 +
|-
 +
| 000014d1
 +
| 000014d1
 +
| 000014d1
 +
| [[Home Menu]] badge
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001632
 +
| Fullblox / Stretchmo
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001646
 +
| Pokémon Rumble World
 +
|
 +
|-
 +
| 00001648
 +
| 00001648
 +
| 00001648
 +
| Pokémon Sun<br>Pokémon Moon
 +
|
 +
|-
 +
| 0000165c
 +
| 0000165c
 +
| 0000165c
 +
| [[Home Menu]] saved theme layouts
 +
|
 +
|-
 +
| 000016C6
 +
| ?
 +
| 00001678
 +
| Yo-kai Watch
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 000018fa
 +
| Phoenix Wright: Ace Attorney - Spirit of Justice
 +
|
 +
|-
 +
| ?
 +
| 0000198e
 +
| 0000198f
 +
| Animal Crossing: New Leaf - Welcome amiibo
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001a05
 +
| Super Mario Maker
 +
|
 +
|-
 +
| ?
 +
| ?
 +
| 00001a2e
 +
| Swapdoodle
 +
|
 +
|}
 +
 
 +
== NAND Shared Extdata ==
 +
{| class="wikitable" border="1"
 +
|-
 +
!  ExtdataID
 +
!  Description
 +
|-
 +
| 0xe0000000
 +
| Home Menu attempts to open this archive during boot, if [[FS:OpenArchive]] doesn't return an error Home Menu seems to then launch the [[System Transfer]] application. Home Menu doesn't actually use this archive at all except for checking whether it exists.
 +
|-
 +
|  0xf0000001
 +
|  NAND JPEG/MPO files and phtcache.bin from the camera application are stored here. This also contains UploadData.dat.
 +
|-
 +
|  0xf0000002
 +
|  NAND M4A files from the sound application are stored here
 +
|-
 +
|  0xf0000009
 +
|  Used for [[BOSS_Services|SpotPass]] content storage for [[News_Services|notifications]].
 +
|-
 +
|  0xf000000b
 +
|  Contains idb.dat, idbt.dat, gamecoin.dat, ubll.lst, CFL_DB.dat, and CFL_OldDB.dat. These files contain cleartext [[Mii_Maker|Miis]] and some data relating (including cached ICN data) to Play/Usage Records.
 +
|-
 +
|  0xf000000c
 +
|  Contains bashotorya.dat and bashotorya2.dat.
 +
|-
 +
|  0xf000000d
 +
|  Home Menu SpotPass content data [[BOSS_Services|storage]].
 +
|-
 +
|  0xf000000e
 +
|  Contains [[VersionList|versionlist.dat]], used by Home Menu for the software update notification added with [[7.0.0-13]].
 +
|}
 +
 
 +
==== Shared Extdata 0xf000000b gamecoin.dat ====
 +
{| class="wikitable" border="1"
 +
|-
 +
!  Offset
 +
!  Size
 +
!  Description
 +
|-
 +
| 0x0
 +
| 0x4
 +
| Magic number: 0x4F00
 +
|-
 +
| 0x4
 +
| 0x2
 +
| Total Play Coins
 +
|-
 +
| 0x6
 +
| 0x2
 +
| Total Play Coins obtained on the date stored below. When the below date does not match the current date, this field is reset to zero, then the date(and other fields) are updated. Once this value is >=10, no more Play Coins can be obtained until the current date changes.
 +
|-
 +
| 0x8
 +
| 0x4
 +
| Total step [[PTM:GetTotalStepCount|count]] at the time a new Play Coin was obtained.
 +
|-
 +
| 0xC
 +
| 0x4
 +
| Step count for the day the last Play Coin was obtained, for that day's step count(same as the step count displayed by home-menu when this file was updated).
 +
|-
 +
| 0x10
 +
| 0x2
 +
| Year
 +
|-
 +
| 0x12
 +
| 0x1
 +
| Month
 +
|-
 +
| 0x13
 +
| 0x1
 +
| Day
 +
|}
 +
 
 +
The above date stores the last time new Play Coin(s) were obtained. The contents of this file is updated by home-menu. [[PTM:GetTotalStepCount]] is not checked constantly, after home-menu boot this is only checked when waking from sleep-mode. Each time home-menu updates the contents of this file, home-menu will set the Play Coin total to 300 if it's higher than the 300 Play Coin limit.
 +
 
 +
[[Home Menu]] loads this file / opens this archive during [[Home Menu|startup]]. When accessing this file fails, like when the file/archive is corrupted(or at least on older system-versions), the result is a brick due to Home Menu using [[SVC|svcBreak]]. [[User:Yellows8|Yellows8]] bricked a 3DS this way due to corruption via invalid [[FSFile:Write]] flush flags. When opening this extdata archive(0xf000000b) fails, Home Menu executes svcBreak.
 +
 
 +
==== Shared Extdata 0xf000000b ubll.lst ====
 +
List of blocked users.
 +
 
 +
Empty space is filled with 0xC-long sequences of 00 00 ... 07
 +
 
 +
== Tools ==
 +
* [https://github.com/wwylele/3ds-save-tool 3ds-save-tool] - Extract/verifies extdata
Meleemeister
68

edits

Retrieved from "https://www.3dbrew.org/wiki/Special:MobileDiff/307...22521"