GPU/Pitfalls

2025-11-01T21:39:06Z

Wyatt: Document L4-format texture crash

This page collects some oddities and pitfalls of the PICA GPU which is used in the 3DS.

[[Category:GPU]]

== Internal Registers ==

=== Vertex attribute alignment ===

Vertex components which are defined through [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG1|GPUREG_ATTRIBBUFFERi_CONFIG1]] will be accessed aligned by the GPU.
* Vertex attributes will be aligned to their component element size.
* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.
* The stride which is passed to the GPU should be passed unaligned.

=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===

The vertex stride set in [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG2|GPUREG_ATTRIBBUFFERi_CONFIG2]] must match the actual size of the vertex contained in the buffer or the PICA will freeze or it won't draw anything.

If you want to use a different stride you have to pad the data accordingly with padding attributes.

=== Output mapping in GPUREG_SH_OUTMAP_MASK ===

The output masking in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK|GPUREG_SH_OUTMAP_MASK]] influences how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi|GPUREG_SH_OUTMAP_Oi]] map to outputs in the shader.

If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi|GPUREG_SH_OUTMAP_Oi]] registers is consumed.
[[GPU/Internal_Registers#GPUREG_SH_OUTMAP_TOTAL|GPUREG_SH_OUTMAP_TOTAL]] configures the number of used consecutive slots in the outmap.

Example:

{| class="wikitable" border="1"
! Register
! Value
! Meaning
|-
|GPUREG_SH_OUTMAP_TOTAL
|0x00000002
|2 outputs enabled
|-
|GPUREG_SH_OUTMAP_MASK
|0x00000011
|o0 enabled, o4 enabled
|-
|GPUREG_SH_OUTMAP_O0
|0x03020100
|o0 = pos.xyzw
|-
|GPUREG_SH_OUTMAP_O1
|0x0B0A0908
|o4 = color.rgba
|-
|GPUREG_SH_OUTMAP_O2
|...
|(unused)
|}

=== TEXUNIT1 Corruption and Instability ===

If an L4-formatted texture (or likely any 4-bit texture format) located within VRAM is bound to TEXUNIT1, the PICA can become unstable, usually freezing after a seemingly random amount of time. If that texture is 8x16 or larger, the GPU will also display corrupt visuals until the console is rebooted; this will even persist in other software. Unbinding the texture makes the console stable again, but doesn't fix the visuals, though the severity of the corruption is slightly reduced because the exact results also depend on the texture data. Notably, this does not seem to occur with other TEXUNITs or with textures located in FCRAM. This could use more investigation.

Writer's Note: I struggled for days to reproduce this. I happened across it in a real application, and swapping between L4 and L8 affected the issue with 100% consistency. I then went to make this write-up months later and made a test program to reproduce the bug, which worked as expected. Like a doofus I rewrote the program a bit to test a bit more thoroughly, somehow caused the bug to stop occurring, and haven't been able to reproduce it since. Needs further investigation.

== Shaders ==

=== Configued Output components must be written exactly once ===

Each configured output component has to be written exactly once or the PICA freezes.

=== MOVA instructions can't be adjacent ===

Having 2 consecutive MOVA instructions will freeze the PICA. This can be relaxed by placing a NOP between 2 MOVAs or by rearranging the code.

3dbrew - User contributions [en]

GPU/Pitfalls