Nintendo 64 Architecture
The Nintendo 64
Released on 23/06/1996 in Japan, 29/09/1997 in America and 01/03/1997 in Europe
Showing revision 'NUS-CPU-03'.
Later ones reduced the number of chips required for AV encoding
Disk Drive connector is found on the back

A quick introduction

Nintendo’s goal was to give players the best graphics possible, for this it will partner with one of the biggest players in computer graphics to produce the ultimate graphics chip.

The result was a nice-looking console for the family… and a 500-page manual for the developer.

Don’t worry, I promise you this article will not be that long… Enjoy!


CPU

The main processor is a NEC VR4300 that runs at 93.75 MHz, it’s a binary-compatible version of Silicon Graphics’ MIPS R4300i that features:

An internal 64-bit FPU is also included in this package, the CPU identifies it as a co-processor (COP1) although the unit is fitted next to the ALU and it’s only accessed through the ALU pipeline, meaning there’s no co-processing per se.

Simplified memory access

The way RAM is assembled follows the unified-memory architecture or ‘UMA’ where all available RAM is centralised in one place only and all components that need to use will access this same location. The component arbitrating its access is, in this case, the GPU.

The reason for choosing this design comes to the fact that it saves a considerable amount of production costs while, on the other side, it increments access latency.

No DMA controller?

Due to the unified memory architecture, the CPU no longer has direct access to RAM, so the GPU will be providing DMA functionality as well.

RAM Available

Apart from the UMA, the way of addressing RAM is a little bit complicated, so I’ll try to keep it simple, here it goes…

The system physically contains 4.5 MB of RAM, however it’s connected using a 9-bit bus where the 9th bit can only be accessed by the GPU (more details later). As a consequence, every component except the GPU will only find up to 4 MB.

The type of RAM fitted in the board is called Rambus DRAM or ‘RDRAM’ for short, this was just another design that competed against SDRAM on becoming the next standard. RDRAM is connected in serial (where transfers are done one bit at a time) while SDRAM uses a parallel connection (transfers multiple bits at a time).

RDRAM’s latency is directly proportional to the number of banks installed and as a consequence, with the amount of RAM this system has, the resulting latency is significant.

By contrast, the amount of available RAM on this console can be expanded by installing the Expansion Pak accessory: A fancy-looking small box that includes 4.5 MB. Curiously enough, the RAM bus must be terminated, so the console always shipped with a terminator (called Jumper Pak) fitted in the place of the Expansion Pak. Now, you may ask, what would happen if you switch on the console without any Pak installed? Literally nothing, you get a blank screen!


Graphics

The core of the graphics reside on a huge chip designed by Silicon Graphics called Reality Co-Processor running at 62.5 MHz. This package contains a lot of circuitry so don’t worry if you find it difficult to follow, the graphics sub-system has a very complex architecture!

Anyway, this chip is divided into three main modules, two of them are used for graphics processing:

Reality Signal Processor

RSP Diagram
Architecture of the RSP

Also known as RSP, it’s just another CPU package composed of:

  • The Scalar Unit: A MIPS R400-based CPU which implements a subset of the R400 instruction set.
  • The Vector Unit: A co-processor that performs vector operations with 32 128-bit registers. Each register is sliced in eight parts to operate eight 16-bit vectors at once (just like SIMD instructions on conventional CPUs).
  • The System Control: Another co-processor that provides DMA functionality and controls its neighbour module, the RDP (more about it later on).

In order to operate this module, the CPU stores in RAM a series of commands called Display list along with the data that will be manipulated, then the RSP reads the list and applies the required operations on it. The available features include:

  • Geometry transformations.
  • Clipping & Culling (Removing unnecessary & unseen polygons).
  • Lighting.

This seems straightforward, but how does it perform these operations? Well, here’s the interesting part: Unlike its competitors (PS1 and Saturn), the geometry engine is not hard-wired. Instead, the RSP contains some memory (4 KB for instructions and 4 KB for data) to store microcode, a small program, with no more than 1000 instructions, that implements the graphics pipeline. In other words, it directs the Scalar Unit on how it should operate our graphics data. The microcode is fed by the CPU during runtime.

Nintendo provided different microcodes to choose from and, similarly to the SNES’ background modes, each one balances the resources differently.

Reality Display Processor

RDP Diagram
Architecture of the RDP

After the RSP finished processing our polygon data, it will start sending rasterisation commands to the next module, the RDP. These commands are either sent using a dedicated bus called XBUS or through main RAM.

The RDP is just another processor (this time with fixed functionality) that includes multiple engines used to apply textures over our polygons and project them on a 2D bitmap.

It can process either triangles or rectangles as primitives, the latter is useful for drawing sprites. The RDP’s rasterisation pipeline contains the following blocks:

  • A Rasteriser: Allocates the initial bitmap that will serve as frame-buffer.
  • A Texture Unit: Basically applies textures to our polygons using 4 KB of dedicated memory (called ‘TMEM’) allowing up to eight tiles to be used for texturing. It can also performs the following operations on them:
    • 4-to-1 bilinear filtering: Smooths out our textures.
    • Perspective correction: Improves the coordinate precision of the textures.
  • A Colour Combiner: Mixes and interpolates multiples layers of colours (for instance, to apply shaders).
  • A Blender: Mixes pixels against the current frame-buffer in order to apply translucency, anti-aliasing, fog, dithering and z-buffering, the latter one is critical to efficiently deduce the depth of each polygon in a 3D space (instead of needing software-based polygon sorting which drains a lot of CPU resources).
  • A Memory interface: Used by multiple blocks to read and write the current frame-buffer in RAM and/or fill the TMEM.

The RDP provides four modes of functioning, each mode combines these blocks differently in order to optimise specific operations.

Since this module is constantly updating the frame-buffer in RAM, it uses an special addressing mode: Remember the unusual 9-bit addressing? The ninth bit is reserved for special calculations (like z-buffering) which can only be accessed using the Memory interface.


The resulting frame-buffer will be captured by the video encoder and sent through the video signal. The theoretical maximum capabilities are 24 bit colour depth (16.8 million colors) and 640x480 resolution (or 720x576 in the PAL region).

I mention it as ‘theoretical’ since using the maximum capabilities can be resource-hungry, so programmers will tend to use lower stats in order to free up enough resources for other services.

Quick demo

Let’s put all the previous explanations into perspective, for that I’ll borrow Nintendo’s Super Mario 64 to show, in a nutshell, how a frame is composed:

Models and Lighting

RSP Diagram
Primitive view of our scene
In order to save polygons, some characters are designed to be sprites (using quads)

To start with, our 3D models are located in the cartridge ROM, but in order to keep a steady bandwidth, we need to copy them to RAM first.

Then it’s time to build a scene using our models, the CPU could do it by itself but it may take ages, so the task is delegated to the RCP. The CPU will instead send orders to the RCP, this is done by carrying out these tasks:

  1. Compose the Display List that contains the operations to be carried out by the RSP and store it in RAM.
  2. Point the RSP where the display lists are.
  3. Send microcode to the RSP to ‘activate’ the Scalar Unit.

Afterwards, the RSP will start performing the first batch of tasks and the result will be sent to the RDP in the form of rasterisation commands.

Textures and Projection

RSP Diagram
Rendered frame (Tada!)

So far we managed to process our data and apply some effects on it, but we still need to:

  • Apply textures and other effects.
  • Build a 2D map from it, so we can display it on a screen.

As you may guess, these tasks will be performed by the RDP. The CPU will provide the data (such as tiles) by placing it on RAM, this module has a fixed pipeline but we can select an optimal mode of operation to improve the frame-rate.

When the RDP finished it will write the last bitmap to the frame-buffer area in RAM, then the CPU will transfer the frame-buffer to the Video Interface (preferably using the DMA) which is then sent to the Video Encoder for display.


Designs

Here are some examples of previous 2D characters for the Super Nintendo that have been redesigned for the new 3D era, they are interactive so I encourage you to check them out!

3D model
The Legend of Zelda: Ocarina of Time (1998)
785 triangles
3D model
Kirby 64: The Crystal Shards (2000)
516 triangles

Secrets and limitations

SGI clearly invested a lot of technology into this system, however this was a console meant for the household and as such it had to keep its cost down. Some hard decisions resulted in difficult challenges for programmers:

Pipeline Stalls

Due to the huge number of components and operations in the graphics pipeline, the RCP ended up being very susceptible to stalls: An undesirable situation where sub-components keep idling for considerable periods of time because the required data is delayed at the back of the pipeline.

This will always result in performance degradation and is up to the programmer to avoid them. Although to make things easier, some CPUs such as the Scalar Unit implement a feature called Bypassing which enables to execute similar instructions at a faster rate by bypassing some execution stages that can be skipped. For example, if we have to compute sequential ‘add’ instructions there’s no need to write the result back to a register and then read it back every time each ‘add’ is finished, we can instead keep using the same register for all additions and do the write back once the last ‘add’ is completed.

Texture memory

Inside the RDP there are 4 KB Texture memory available to be used as Texture Cache, its main goal is to avoid stalling read cycles from RAM. Unfortunately, in practice 4 KB happened to be insufficient for high-resolution textures.

As a result, some games used solid colours with Gouraud shading (like Super Mario 64) and others relied on pre-computed textures (for example, where multiple layers had to be mixed).


Audio

Before we go into the details, let’s define the two endpoints of the audio sub-system:

Now, how do we connect both ends? Consoles normally include a dedicated audio chip that does the work for us. Unfortunately, the Nintendo 64 doesn’t have such dedicated chip, so this task is distributed across these components:

The resulting data is, as expected, waveform data. This is then sent to the Audio Interface or ‘AI’ block which will then transfer it to the digital-to-analog converter. The resulting waveform contains two channels (since our system is stereo) with 16-bit resolution each.

VDP Diagram
Overview of how the audio pipeline is often programmed

Secrets and limitations

Because of this design, the constraints will depend on the implementation:


Operating System

Similar to the PS1 and Saturn, N64 games are written for bare-metal, however there are no BIOS routines available to simplify some operations. As a substitute, games embed small OS that provides a fair amount of abstraction to efficiently handle the CPU, GPU and I/O.

This is not the conventional desktop OS that we may imagine at first, it’s just a micro-kernel with the smallest footprint possible that provides the following functionality:

The kernel is automatically embedded by using Nintendo’s libraries, additionally, if programmers decide not to include one of the libraries, the respective portion of the kernel is skipped to avoid cartridge space being wasted.

Input/Output

As you know by now, I/O is not directly connected to the CPU, so the RCP’s third module (which I haven’t mentioned until now) serves as a I/O interface, it basically communicates with the CPU, controllers, game cartridge and Audio/Video DACs.


Games

Nintendo held on to the cartridge as medium for storage and as a consequence, games enjoyed higher bandwidths (between 5-50 MB/s depending on the ROM’s speed) while being more expensive to produce. The biggest cartridge found in the market has 64 MB.

Inside cartridges manufacturers may include extra memory (in the form of EEPROM, flash or SRAM with a battery) to hold saves, however this is not a strong requirement any more since certain accessories could be used to store saves as well.

Accessories

The Nintendo 64 controller included a connector used to plug in accessories, some of them are:

All accessories connected to the controller are managed by the Peripheral Interface.

Apart from that, this console included a special connector at the bottom of its motherboard which was meant to be used by the yet-unreleased Disk drive, some sort of an ‘extra floor’ that contained a proprietary disk reader, the drive was only released on Japan nonetheless and eventually cancelled for the rest of the world.

Source Development Kit

In general, development was mainly done in C, assembly was also used to achieve better performance. While this system contained a 64-bit instruction set, 64-bit instructions were rarely used since in practice, 32-bit instructions happened to be faster to execute and required half the storage.

Libraries contained several layers of abstractions in order to command the RCP, for example, structs like the Graphics Binary Interface or ‘GBI’ were designed to assemble the necessary Display lists more easily, the same applied for audio functions (its struct was called Audio Binary Interface or ‘ABI’).

In terms of microcode development, Nintendo already provided a set of microcode programs to choose from, however if developers wanted to customise it, that would indeed be a challenging tasks: The Scalar Unit instruction set wasn’t initially documented (at the request of Nintendo, of course), later on the company changed its position and SGI finally released some documentation for microcode programming.

Hardware used for development included workstations supplied by SGI, like the Indy machine which came with an extra daughterboard called ‘U64’ that emulates the retail console. Tools were supplied for Windows computers as well.

Other third-party tools consisted in cartridges containing wide cables that connected to the workstation, this cartridge was fitted on a normal Nintendo 64 and included internal circuitry to redirect the read requests from the console to the workstation’s RAM. Debugging was carried out by transferring a copy of the game to RAM and then, when the console was switched on, it would start reading from there.


Anti-piracy / Region Lock

The anti-piracy system is a continuation of the SNES’ CIC. As you know, bootleg detection and region locking is possible thanks to the CIC chip (which must be present in every authorised game cartridge), the Nintendo 64 improved this system by requiring different games to have an specific variant of the CIC chips in order to make sure the cartridge was not a counterfeit or contained a CIC clone, the Peripheral Interface or ‘PIF’ would do checksum checks at the start and during gameplay to supervise current CIC installed on the cartridge.

If by any reason the PIF considers the current cartridge is not valid, it will then induce the console in a permanent freeze.

Region-locking was done by slightly altering the shape of the cartridge between different regions so the user can’t physically insert the game on a N64 from a different region.

Overall, there was not too much concern regarding piracy thanks to the use of cartridge medium, although price on games were three times higher than CD based ones.

Unused ports

As silly as it may seem, Nintendo left one door opened: The Disk Drive port.

A few companies reversed engineered the interface in order to develop their own hardware, and some of the resulting products became a concern for piracy.

I guess the one worth mentioning is the Doctor v64, this device has the same shape as the Disk Drive port but included a CD-ROM drive that’s used to clone the contents of the cartridge to a CD, the opposite (reading Roms from a CD) is also possible.

Emulation

When I was a kid I used to play some N64 games on a Pentium II machine using an emulator, it wasn’t that bad but I wondered now how the freck was it able to happily emulate a complex 64-bit machine since, among other things, my PC barely had enough RAM to keep the integrated video alive.

The truth is, while reproducing the architecture of this console can be complex, things like microcode will give a hint of what the console is trying to do, and since emulators don’t have to be cycle-accurate, they can apply enough optimisations to provide more performance in exchange for real emulation.
Another example are the 64-bit instructions, since games barely used them, emulation speed would hardly be hit when running on a 32-bit host machine.


That’s all folks

I have to say, this article may be the longest one I’ve ever written, but hopefully you found it a nice read!

I’ll probably take the following days to tide up some things on the website instead of starting to write the next article.

Anyway, if you enjoy my articles and would like to help, please take a look here. If you have any comments or suggestions, feel free to email me here.

Until next time!
Rodrigo


Sources / Keep Reading

General

CPU

Graphics / Audio

Games

Anti-piracy

Photography

Bonus


Contributing

This article is part of the Architecture of Consoles series. If you found it interesting please consider donating, your contribution will be used to get more tools and resources that will help to improve the quality of current articles and upcoming ones.

Paypal

Changelog

## 2019-10-29

- Added some 3d models to fiddle with

## 2019-09-17

- Added a quick introduction
- Corrected some explanations

## 2019-09-12

- Released to the public, yay

Rodrigo Copetti

Rodrigo Copetti

Hope you enjoyed the article! If you want to know more about the author tap here and if you'd like to support him tap here instead

rss facebook twitter reddit