---
title: 'Why Neural Networks can learn (almost) anything'
source: 'https://youtube.com/watch?v=0QczhVg5HaI'
video_id: '0QczhVg5HaI'
date: 2026-07-28
duration_sec: 630
---

# Why Neural Networks can learn (almost) anything

> Source: [Why Neural Networks can learn (almost) anything](https://youtube.com/watch?v=0QczhVg5HaI)

## Summary

This video explains how neural networks function as universal function approximators, using the Mandelbrot set as a visual example. It covers the core concepts of neurons, activation functions, and how combining simple building blocks allows networks to learn complex patterns.

### Key Points

- **Neural network learning visualization** [00:00] — An artificial neural network is shown learning the shape of the Mandelbrot set, illustrating complexity all the way down.
- **What is a function** [00:26] — A function is a system of inputs and outputs; knowing the function allows calculating the correct output for any input.
- **Reverse engineering a function** [00:51] — If we only have some input-output pairs, we can try to approximate the underlying function, even with noisy data.
- **Neural network as function approximator** [01:35] — A neural network is a function approximator that learns to map inputs to outputs from data.
- **Network structure** [02:25] — The network consists of layers of neurons; each neuron takes inputs, multiplies by weights, adds a bias, and applies an activation function.
- **Linear functions limitation** [03:47] — Combining linear functions only yields another linear function, which cannot approximate complex patterns.
- **Activation function (ReLU)** [04:28] — Using a non-linear activation function like ReLU allows neurons to build complex, non-linear approximations.
- **Backpropagation** [05:29] — Backpropagation is the algorithm that automatically tunes weights and biases to improve the approximation.
- **Universal approximation theorem** [06:37] — Neural networks are universal function approximators: they can approximate any function to any desired precision.
- **Applications** [07:05] — Any task that can be expressed as a function (e.g., image classification, translation) can be learned by a neural network.
- **Turing completeness** [08:02] — Under certain assumptions, neural networks are Turing complete, meaning they can simulate any algorithm.
- **Limitations** [08:32] — Practical limitations include finite neurons, data requirements, and the difficulty of finding optimal parameters.

### Conclusion

Neural networks are powerful function approximators that can learn almost any pattern given sufficient data and computational resources, but they are not magic and have practical limitations.

## Transcript

you are currently watching an artificial
neural network learn
in particular it's learning the shape of
an infinitely complex fractal known as
the mandelbrot set
this is what that set looks like
complexity all the way down
now in order to understand how a neural
network can learn the mandelbrot set
really how it can learn anything at all
we will need to start with a fundamental
mathematical concept
what is a function
informally a function is just a system
of inputs and outputs numbers in numbers
out
in this case you input an x and it
outputs a y
you can plot all of a function's x and y
values in a graph where it draws out a
line what is important is that if you
know the function you can always
calculate the correct output y given any
input x
but say we don't know the function and
instead only know some of its x and y
values we know the inputs and outputs
but we don't know the function used to
produce them
is there a way to reverse engineer that
function that produced this data
if we could construct such a function we
could use it to calculate a y value
given an x value that is not in our
original data set this would work even
if there was a little bit of noise in
our data a little randomness we can
still capture the overall pattern of the
data and continue producing y values
that aren't perfect but close enough to
be useful what we need is a function
approximation and more generally a
function approximator
that is what a neural network is
this is an online tool for visualizing
neural networks and i'll link it in the
description below this particular
network takes two inputs x1 and x2 and
produces one output technically this
function would create a
three-dimensional surface but it's
easier to visualize in two dimensions
this image is rendered by passing the x
y coordinate of each pixel into the
network which then produces a value
between negative one and one that is
used as the pixel value these points are
our data set and are used to train the
network when we begin training it
quickly constructs a shape that
accurately distinguishes between blue
and orange points building a decision
boundary that separates them it is
approximating the function that
describes the data it's learning and is
capable of learning the different data
sets that we throw at it
so what is this middle section then well
as the name implies this is the network
of neurons each one of these nodes is a
neuron which takes in all the inputs
from the previous layer of neurons and
produces one output which is then fed to
the next layer
inputs and outputs sounds like we're
dealing with a function
indeed a neuron itself is just a
function one that can take any number of
inputs and has one output each input is
multiplied by a weight and all are added
together along with bias the weights and
bias make up the parameters of this
neuron values that can change as the
network learns
to keep it easy to visualize we'll
simplify this down to a two-dimensional
function with only one input and one
output
now neurons are our building blocks of
the larger network building blocks that
can be stretched and squeezed and
shifted around and ultimately work with
other blocks to construct something
larger than themselves the neuron as
we've defined it here works like a
building block it is actually an
extremely simple linear function one
which forms a flat line or plane when
there's more than one input
with the two parameters the weight and
bias we can stretch and squeeze and move
our function up and down and left and
right
as such we should be able to combine it
with other neurons to form a more
complicated function one built from lots
of linear functions
so let's start with a target function
one we want to approximate i've
hard-coded a bunch of neurons whose
parameters were found manually and if we
weight each one and add them up as would
happen in the final neuron of the
network we should get a function that
looks like the target function
well that didn't work at all what
happened
well if we simplify our equation
distributing weights and combining like
terms we end up with a single linear
function
turns out linear functions can only
combine to make one linear function this
is a big problem because we need to make
something more complicated than just a
line we need something that is not
linear a non-linearity
in our case we will be using a relu a
rectified linear unit we use it as our
activation function meaning we simply
apply it to our previous naive neuron
this is about as close as you can get to
a linear function without actually being
one and we can tune it with the same
parameters as before
however you may notice that we can't
actually lift the function off of the
x-axis which seems like a pretty big
limitation
well let's give it a shot anyway and see
if it performs any better than our
previous attempt
we're still trying to approximate the
same function and we're using the same
weights and biases as before but this
time we're using a value as our
activation function
and just like that the approximation
looks way better unlike before our
function cannot simplify down to a flat
linear function if we add the neurons
one by one we can see the simple value
functions building on one another and
the inability for one neuron to lift
itself off the x-axis doesn't seem to be
a problem many neurons working together
overcome the limitations of individual
neurons
now i manually found these weights and
biases but how would you find them
automatically the most common algorithm
for this is called back propagation and
is in fact what we're watching when we
run this program
it essentially tweaks and tunes the
parameters of the network bit by bit to
improve the approximation and the
intricacies of this algorithm are really
beyond the scope of this video i'll link
some better explanations in the
description
now we can see how this shape is formed
and why it looks like it's made up of
sort of sharp linear edges it's the
nature of the activation function we're
using
we can also see why if we use no
activation function at all the network
utterly fails to learn we need those
non-linearities
so what if we try learning a more
complicated data set like this spiral
let's give it a go
seems to be struggling a little bit to
capture the pattern no problem if we
need a more complicated function we can
add more building blocks more neurons
and layers of neurons and the network
should be able to piece together a
better approximation something that
really captures the spiral
it seems to be working
in fact no matter what the data set is
we can learn it that is because neural
networks can be rigorously proven to be
universal function approximators they
can approximate any function to any
degree of precision you could ever want
you can always add more neurons
this is essentially the whole point of
deep learning because it means that
neural networks can approximate anything
that can be expressed as a function a
system of inputs and outputs
this is an extremely general way of
thinking about the world the mandelbrot
set for instance can be written as a
function and learned all the same this
is just a scaled-up version of the
experiment we were just looking at but
with an infinitely complex data set
we don't even really need to know what
the manual brought set is the network
learns it for us and that's kind of the
point if you can express any intelligent
behavior any process any task as a
function then a network can learn it for
instance your input could be an image
and your output a label as to whether
it's a cat or a dog or your input could
be text in english and your output a
translation to spanish you just need to
be able to encode your inputs and
outputs as numbers but computers do this
all the time images video text audio
they can all be represented as numbers
and any processing you may want to do
with this data so long as you can write
it as a function can be emulated with a
neural network
it goes deeper than this though under a
few more assumptions neural networks are
provably turing complete meaning they
can solve all of the same kinds of
problems that any computer can solve
an implication of this is that any
algorithm written in any programming
language can be simulated on a neural
network but rather than being manually
written by a human it can be learned
automatically with a function
approximator
neural networks can learn anything
okay that is not true first off you
can't have an infinite number of neurons
there are practical limitations on
network size and what can be modeled in
the real world
i've also ignored the learning process
in this video and just assumed that you
can find the optimal parameters
magically how you realistically do this
introduces its own constraints on what
can be learned
additionally in order for neural
networks to approximate a function you
need the data that actually describes
that function if you don't have enough
data your approximation will be all
wrong it doesn't matter how many neurons
you have or how sophisticated your
network is you just have no idea what
your actual function should look like
it also doesn't make a lot of sense to
use a function approximator when you
already know the function you wouldn't
build a huge neural network to say learn
the mandelbrot set when you can just
write three lines of code to generate it
unless of course you want to make a cool
background visual for a youtube video
there are countless other issues that
have to be considered but for all these
complications neural networks have
proven themselves to be indispensable
for a number of really rather famously
difficult problems for computers
usually these problems require a certain
level of intuition and fuzzy logic that
computers generally lack and are very
difficult for us to manually write
programs to solve
fields like computer vision natural
language processing and other areas of
machine learning have been utterly
transformed by neural networks
and this is all because of the humble
function a simple yet powerful way to
think about the world and by combining
simple computations we can get computers
to construct any function we could ever
want
neural networks can learn almost
anything
[Music]