# Neural Radiance Fields (NeRFs), Simply Explained #76 Nov 10, 2025 Architecture https://www.aibutsimple.com/p/neural-radiance-fields-nerfs-simply-explained Imagine taking a picture in the room you’re in right now. Take one step to the right and snap another. The room in this new picture will look different from the first. Take two steps forward and turn around. Now you’ve got a picture that includes objects and features that may not have been in the first two. But sometimes looking at a picture isn’t enough; we need more than just one angle to determine what an object or scene truly captures. An extra dimension, perhaps. Neural Radiance Fields (NeRFs) actualize this concept, bringing 2D images into 3D spatial features, which was first introduced by a [paper](https://arxiv.org/pdf/2003.08934?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) written by Ben Mildenhall and his team in 2020. NeRF learns scenes that were never photographed by utilizing a continuous function that describes how light (radiance) interacts with and passes through every point in a 3D space. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9d7fd8df-8ae4-4521-8235-d54b25a8d2e0/Copy_of_The_red_dog_barks.__11_.png) Instead of relying on triangle meshes like traditional 3D graphics renderers, it sculpts objects by treating the scene as a volume, where each point of space has two key properties: density (how opaque/solid the point is) and color (the RGB values emitted from that point in a given viewing direction). --- [![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/3d07933c-56f3-4e33-8801-be4127e06d2c/ChatGPT_V1.jpg)](https://offers.hubspot.com/using-chatgpt-at-work?utm_medium=email-media-newsletter&utm_source=XIAEVJHTGJ&utm_campaign=creator&utm_content=beehiiv&utm_term=version-i&_bhiiv=opp_68e51678-7867-4043-9f5d-4408b4ec2c2b_b942af4d) ### Want to get the most out of ChatGPT? ChatGPT is a superpower if you know how to use it correctly. Discover how [HubSpot's guide to AI](https://offers.hubspot.com/using-chatgpt-at-work?utm_medium=email-media-newsletter&utm_source=XIAEVJHTGJ&utm_campaign=creator&utm_content=beehiiv&utm_term=version-i&_bhiiv=opp_68e51678-7867-4043-9f5d-4408b4ec2c2b_b942af4d) can elevate both your productivity and creativity to get more things done. Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI. [Download the free guide.](https://offers.hubspot.com/using-chatgpt-at-work?utm_medium=email-media-newsletter&utm_source=XIAEVJHTGJ&utm_campaign=creator&utm_content=beehiiv&utm_term=version-i&_bhiiv=opp_68e51678-7867-4043-9f5d-4408b4ec2c2b_b942af4d) --- ### NeRF Architecture NeRFs rely on deep fully-connected neural networks without any convolutional layers, which are otherwise known as multilayer perceptrons (MLPs). MLPs are useful in the context of NeRFs, as the images are not being treated like usual. Unlike Convolutional Neural Networks (CNNs), which exploit the gridlike structure of images by analyzing neighboring pixels to understand textures, NeRFs use a different type of data in their input. They take coordinates: (x, y, z, viewing_angle). The x, y, and z coordinates are with respect to the scene, preserving the fact that the corner of a table is in the same position, regardless of the camera angle. The viewing_angle is split up into two coordinates, theta and phi, which represent how the camera angle captures the scene. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/fff22946-b466-4947-b0de-420e67ebd4fd/Copy_of_The_red_dog_barks.__9_.png) Note: In some implementations of NeRF, the theta and phi coordinates are replaced with (dx, dy, dz), which function the same if you treat them as looking along their respective axes. Example: (1,0,0) means the camera looks along the positive x-axis. Unlike CNNs, which use shared filters that slide across a grid, MLPs learn a function that maps any input coordinate (x, y, z) to an output (color, density). This means the network can represent arbitrary 3D relationships: the point at (1, 2, 3) can have a completely different density than (1, 2, 4), even though they're "neighbors" in coordinate space. This flexibility is essential for representing continuous 3D scenes where there's no inherent grid structure. Inherently, the “fully-connectedness” of the neural network gives way for the continuous function, as each node in a particular layer will connect to every node in the next one. This allows for the neural network to abstractly detect spatial patterns in 3D, since any and all information from the previous layer contributes to the output of the next layer. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d0dbb6ed-cd18-4cd9-a565-bf4ce3483c47/Copy_of_The_red_dog_barks.__12_.png) While NeRFs do a great job at creating accurate representations as a result of their architecture, note that a blatant consequence reveals itself: expensive compute power. For a neural network with 256 nodes in each layer, that means that between layers, there are over 65,000 connections, and this scales quadratically with width. We’ll talk more in-depth about the computational expenses later on. ### The Machine Underneath The input to a NeRF is a 5-dimensional (sometimes 6) coordinate. Let’s examine what the output represents: the NeRF takes two stages to process the coordinates. For the neural network represented by the function: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/a69f8381-b140-4017-8298-a348f1ed3cc7/image.png) The first part is looking at the (x,y,z), which outputs the volume density, which is view independent. All this means is that the geometry of an object shouldn’t change in different reference angles. The second part looks at the full coordinate to output the RGB/color values at a specific point, which is view dependent since the color of an object may depend on the lighting at any angle. That latter point is what separates NeRFs, as they account for view-dependent effects, helping create realistic lighting and reflections on object surfaces. To find the inputs, the NeRF essentially “casts rays” from the camera to the point it is observing. This ray will start at the camera position and extend until it “hits something.” While the ray extends, it’ll sample points along the ray and query those into the neural network. To find the color at that point, the function for the ray will be used in the color function: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9b4dbdee-851c-4474-9296-d04c75243669/image.png) In NeRF, the continuous function being modeled is the integral that accumulates color along a ray as it passes through a scene. Formally, this is expressed as the top equation, where the sigma is the density at each point, C(t) is the emitted color, and T(t) is the cumulative transmittance, the probability that light travels from the start of the ray to position t without being absorbed or scattered. This integral represents the continuous accumulation of radiance along the ray, and in practice, NeRF approximates it by sampling discrete points and summing their weighted contributions. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0273f12f-769d-4c5e-aac3-3052219555db/image.png) The transmittance function (T(t)) is itself a continuous function nested inside the integral. It decays exponentially as the ray passes through denser regions, since higher densities make it less likely that light will reach the camera unblocked. Together, these two continuous functions describe how NeRF simulates realistic light transport through a 3D scene, which is later discretized to feed into a finite sum for computational efficiency. The output here is the ability to generate a 2D image of a scene from an angle that was never given to the neural network. The real power in NeRF relies on extensions of it, where loading renders of it using [Marching Cubes](https://en.wikipedia.org/wiki/Marching_cubes?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained), [Instant-NGP](https://arxiv.org/abs/2201.05989?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained), or [TensoRF](https://arxiv.org/abs/2203.09517?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained), allows these 2D images to be interacted with like a 3D model. These applications use NeRF-generated images to construct 3D models, similar to ones you may find in game software or AR/VR. --- ### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png) Supporting us by purchasing a PRO membership will get you: - Exclusive Hands-on Tutorials With Code - Full Math Explanations and Beautiful Visualizations - Ad-Free Reading for an Uninterrupted Learning Experience [Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) --- ### An Even Deeper Dive So far, we’ve discussed how a continuous function takes sampled values along camera rays and converts them into color and density values, which creates the output. However, we can take this one step further to create the fine details of an image. Because the inputs we have been using so far have just been three coordinates, the neural network can’t create complexity beyond what three dimensions lends. For example, curved structures may render as blocky and rough, as there’s not enough information for the network to understand the surface topography. To circumvent this problem, we can represent these (x,y,z) coordinates as sine and cosine functions with different frequencies, where more curvature corresponds to higher frequencies. This technique is called positional encoding. Essentially, we can pick a number of frequencies that we want to “define” the coarseness or smoothness in an image. Each component of the position is then fed into a list of sine and cosine functions that will create distinctions. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/6a5ba4e6-5561-4954-8fb6-2b175b171e30/image.png) As you can see, the more frequencies, the more components that end up being created to be fed to the neural network. This allows the neural network to identify more distinct features. It's like the difference between viewing an image on a low-resolution screen versus a high-resolution display. The higher resolution reveals fine details, sharp edges, and textures that would be blurred together at lower resolution. - Providing these additional frequency components gives the network access to both coarse and fine-grained spatial information. Mathematically, breaking our inputs into specific sine and cosine functions works, as sinusoidal functions are an extremely powerful tool for approximation. Another reason we use sine and cosine is because they’re orthogonal, meaning that they will contribute independent information. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0790ae06-febe-410d-95db-4d1951cbad7c/Copy_of_The_red_dog_barks.__13_.png) These positional encodings can upgrade the quality of the output image and the eventual render by allowing for precise shading and curvature to be added to the scene. ### Limitations of NeRF As described previously, NeRFs are a very computationally expensive technique in visualization. One reason for this is the quadratic scaling, but another reason is the sheer amount of images it must analyze. For a 1024 x 1024 image, it has over 1 million pixels to analyze, which are passed through multiple layers of a neural network. The computations add up quickly, considering that training NeRFs needs upwards of 60-80 images before it can produce a viable new scene. It can take hours just to produce this one image, which drives users away from its time-intensive approach. The good news is that recent advances in this space have greatly improved many aspects of NeRFs. For instance, [DietNeRF](https://arxiv.org/abs/2104.00677?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) and [Zero-Shot NeRF](https://arxiv.org/abs/2312.12122?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) allow for minimal training images to be fed to the neural network while maintaining quality. They incorporate semantic reasoning into the models by having the model use priors to understand the nature of objects instead of reconstructing them each time from “scratch.” Another great example of an advancement is [pixelNeRF](https://arxiv.org/pdf/2012.02190?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) (UC Berkeley 2021), which extracts individual image features. This way, instead of training one neural network for each scene, the neural network is able to generalize as it collects information from many different input images. On large scales, these neural networks can serve to be like LLMs for images, which come in handy for applications of NeRF. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/75249a98-b23b-4805-bc2f-812643da4c50/Copy_of_The_red_dog_barks.__16_.png) This is the result of running a 2D image through NeRF to produce another 2D image. We initialize this with just 10 frequencies, but as you can see, the images are quite similar. The following shows the difference: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/c2952268-1d76-46b4-b523-752ccc39d4c5/Copy_of_The_red_dog_barks.__17_.png) In our [Pro Article](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained), you can develop this program yourself for a 3D object and render a short video of a 360-degree view. A relevant application is real estate. Sellers can take pictures of their property and could generate images from angles that could be difficult to take pictures from. NeRFs with the more advanced technologies could understand the type of image that is being requested and query for it based on what it has already seen in its training data. For example, if you wanted many angles of your kitchen, perhaps at different times of the day, the NeRF could adjust how the light rays hit off surfaces based on prior pictures of other kitchens. Of course, there are limitations that must be taken into consideration. Early NeRFs could not handle motion very well. What this means is that you would hope that everything is still in your image. Otherwise, camera blur will disrupt how the neural network processes that part of the image. ### The Bottom Line NeRFs represent something larger than just creating never-before-seen-angles. They’re a shift from representing “hard coded” physical entities into functions that can be learned and improved over time. Away from clunky meshes that may take hours of hard human labor and closer to computer-aided visualizations that improve passively. Everything in photography has depended on light for centuries, and since 2020, computers have learned to see that light differently: closer to how our own eyes document it. --- Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained): - Sushant Waidande - Sai Krishna Pashapu - Bob Boncella --- If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend! Want to reach 50000+ ML engineers? Let’s work together: [WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com). If you like, you can also donate to our team to push out better newsletters every week! [Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-radiance-fields-nerfs-simply-explained) --- That’s it for this week’s issue of AI, but simple. See you next week! —AI, but simple team