Rasterization
This article is part of a series, you can find the others here:
- Introduction
- Rasterization
- Raytracing
- Raymarching
- Conclusion
As Scratchapixel asserts in their brilliant article on the subject:
“The rasterization rendering technique is surely the most commonly used technique to render images of 3D scenes, and yet, that is probably the least understood and the least properly documented technique of all (especially compared to ray-tracing).”
I’ve certainly found this to be the case during my initial forays into the subject, most of the time all I could find were high level summaries of what the technique does, so let’s start with that.
Basically, rasterization projects 3D shapes onto a 2D canvas, the image on the 2D canvas being what is eventually displayed on your screen.
A 3D scene is made up of 3D shapes which are in turn made of triangles. You can create very complex 3D shapes which are composed of thousands, millions or billions of triangles (I dunno, trillions? Probably…)
You can think of it as a sort of complex origami, each shape is defined by a surface of folded triangles called a mesh. The smaller and more numerous the triangles are the more detail you can add and the smoother the curves will appear.
These triangle meshes can be broken down even further as each triangle is made up of vertices.
Vertices (singular: vertex) are as simple as it gets for a 3D mesh, they are its atoms. They describe a point in 3D space with numbers in 3 coordinantes X(left/right), Z(forward/backwards), Y(up/down) (It can describe a lot more than that but for now we will keep it simple)
“All this well written, informative and entertaining talk about vertices is great, frankly it’s the highlight of my day, but what does it have to do with rasterization?” you ask.
It’s because that is where the rasterization process starts. Aside from the 3D shapes a scene has at least one camera and its associated image plane.
The camera is just like its real world counterpart. It is the point of view from which we can see the scene and it controls position, orientation, field-of-view and so on.
The image plane is a 2D plane in 3D space. It sits between the camera and the rest of the 3D scene (most of the time.) It is the 2D canvas that actually gets drawn on the screen, in other words it is the representation of your screen in 3D space.
It is onto this plane that we will rasterize our image.
To do so we iterate over each vertex in the scene and transform it from 3D space to 2D space using projection matrices. These provide a series of solutions that allow the vertex to step from one type of space to another until it is able to be rendered to your screen.
This transformation can be represented as a ray between the vertex and the camera (though it is not actually a ray)
Note:
I’m honestly just getting to grips with the basic concept of projection matrices but the fundamental ideas can be abstracted quite neatly as casting rays from the vertex to the camera and finding the intersecting pixel.
Scratchapixel has a more detailed breakdown on the process but for the remainder of this article I’ll use the casting of rays as a shorthand for this projection step as it still conveys the core concepts well.
Again, rasterizers don’t actually cast rays from vertices to the camera but this abstraction is useful for this explanation.
For each of these rays we iterate over all of the pixels in our image plane to check if they are intersecting.
If we find a pixel that is intersecting we know we can fill it in with a colour.
From this process we get a selection of coloured pixels.
The above super low resolution version illustrates how the information is transformed from the 3D scene into individual pixels on the 2D plane.
To improve the image, we could up the resolution (which would mean more pixels with which to draw,) add techniques such as anti-aliasing (which creates the illusion of curved lines by blurring the colours between stepped pixels) and so on.
If this seems very simplistic that is because it is. There are many more steps involved; an important one is determining the visibility of each shape in relation to the camera because they may be blocked by other pieces of geometry. I.E. We don’t want to draw our rabbit friend if there is a tree between her and the camera. For now however we are only interested in the broad concepts so we will skip over this part.
I’m more familiar with web-development than building rendering pipelines, so I decided to imagine the rasterizer as a hilariously simplified JavaScript function and it looked like this:
const camera = new Camera(0, 0 ,0);const sphere = new Sphere(0, 0, 1);const scene = new Scene(camera, [sphere])
const { imagePlane } = camera
scene.geometry.forEach((shape) => { shape.vertices.forEach((vertex) => { imagePlane.pixels.forEach((pixel) => { const ray = castRay(vertex, camera); if (intersects(ray, pixel)) { shade(pixel, vertex.color); } }); });});
The outer scope iterates through all the shapes in the scene (for now a simple sphere) and then we iterate through each vertex in a shape and the inner scope iterates through all the pixels, checking for an intersection with the camera’s image plane.
Side note:
You can understand from these nested loops how increasing the complexity of a scene decreases performance. More detail means more triangles and vertices that must be iterated over which in turn must each iterate over all of the vertices and then pixels, performing checks for intersections, visibility and so on. When you throw in shaders, lighting, post-effects and the like, the overhead for graphics processing can be staggering.
Fortunately there are a lot of very clever optimisations that have been developed for rendering pipelines over the years (not to mention massive leaps in raw processing power) and that speeds up this process a lot.
At this point raw vertex counts are rarely the bottleneck in the pipeline, instead other effects will generally account for any drops in frame-rate you may encounter.
Rasterization can be implemented in a bunch of ways but the core concept remains the same. It is relatively cheap and allows us to render complex, bespoke scenery, characters and effects and so it has become the standard in real-time rendering.
But it does have its limitations and that is where the next technique comes into play.