K-d Trees - Computerphile

Really great video and very well described. Tho the outro is a bit weird with two sounds running at the same time suddenly

This is hands down the best explanation I have seen of K-d trees and it is done in a way that captivates attention. Well done and thank you from college students around the world.

reminds me so much of geohashing just slightly different.

Detecting the closest point from the video isn't robust, but the rest of the algorithm is correct. People refer to the wiki page of kdtree for finding nearest neighbors.

A teacher like no other. My university teacher only made this simple concept extra hard to understand.

>Before we continue looking at p...
Stopped watching. I'm outraged!

Sure, it's O(logn) to search through a k-d tree, but what's the time complexity of constructing such a tree. It seems like it would still be close to O(n) if you have to visit a lot of nodes to accurately construct a k-d tree.

Amazing!

That's a neat data structure

I would really like you to do a video on "Bounding Volume Hierarchy".
I think they implemented it for Unreal 5 (nanite).

I am currently doing something which requires me to calculate the distance to coastlines for every single pixel of water to a set of land (which includes the coastlines). This algorithm is something I've used and finding Mike talking about it really made me happy :)

Any suggestion on what algorithm I could try instead?

Detail of what I've done with KD-Tree: Given a reasonable map (2d array) of water and land, from which a set of land and water locations,
1. Build a tree using the entire land locations
2. Make a query for each water location and get the shortest distance

How frequently do people come up with these smart algorithms and structures to reduce time, memory complexity?

Backtracking is a bitch if you're thinking in make a parallel implementation

The search time was reduced at the expense of great spacial complexity (Storing all nodes in a tree structure in memory)

Hi, I noticed that when I try looking for coordinates nearest to the center, it won't give me the ones that I expected. I willl plot my coordinates on a scatterplot, I see that there are coordinates literally on (0.0) but when I try finding the 10 nearest points to the center it will give me points much farther than the ones I see exist on my scatterplot.. Can you explain why that is? Evem if the computer iterates through all the coordinates and finds the child coordinate how is it that it gives me different results than I am expecting, can someone explain?

Mike's incorrect about triangles. In fact, many ray-tracers use K-d trees to store triangles, including Equinox3D. They are a bit slower to construct than a BVH, but faster to traverse.

What it want to find N nearest neighbors....????

Nice explanation

You could create a language to exploit this mechanism and call it K-D Lang.

K-d Trees are used extensively in RTS games, to calculate gameplay stats e.g damage based on range or units attacking closest enemy etc...

nice explanation

In the case of a vast quantity of points, what should you do if you don't expand the tree all the way down? Or do you always add all points to the tree?

I just finished implementing a Bounding Volume Hierarchy for my 4D Game and this video pops up. To build the BVH I use a very similar approach as is used to build the kd tree using the objects centroid as points, but then spliting is stoped once some amount of objects, like 4, are on the leafs, then work from bottom up calculating the combined AABB's of the leafs, and the AABB's of each branch node so that the AABB contains every child (they may overlap).

In the end you have a structure that with a few AABB tests let you decide which objects are colliding with you, in the best case scenario, with 4 childs each node, you only need to do 60 tests to find a object in the middle of 1M others (The tree will have a depth of 15, each level you check for 4 nodes to decide where to continue), but in practice, the nodes may overlap leading to a few more checks.

I'm studying Graph Theory and I find this fascinating.

So it'll work in six dimensions. And what happens if you want to remove one of the dimensions dynamically - so everything on dimensions 2 and 5 doesn't matter?

Hmm, what do we do, if the node on the right of E is not a leaf node? Do we go further down it and how does the algorithm go?

Very interested in computer graphics and the bounding volume hierarchy over here!

How do you pick the point you're going to use to split on? And what happens if you're splitting in the X direction and there's another point with the same X coordinate?

I remember implementing a k-d structure for a geocahing project in university. Got a 19 in that project :D due to suberb preformance.

neat in fact!

The computer processor speed for lidar scanners makes me think about how for the brain, processes emerge and dissolve in parallel in different parts of the brain at different frequency bands like theta (5–8 Hz), alpha (9–12 Hz), beta (14–28 Hz) and gamma (40–80 Hz), where as it might be slower you still have millions of these processors that generate to complete tasks with additional asynchronous verification for accuracy. Not taking into account the conditioning that occurs for natural neural-networks, animal brains are so much better at unconscious data processing despite using slower processors because they have a ton of evolutionary experience that enables practically infinite capability to flexibly allocate further resources for multiprocessing, it all seems to really boil down to how efficiently programmers can diversify tasks and the efficiency of multithreading capabilities for a processor, while still having limitations because processors cannot really support multiprocessing like brains do.

I listen to these while doing other activities. Mostly just because I like Mike's calming British accent

Do a video on octrees!

I just use a hypersphere polytope search.
Much quicker.

The downside with the K-d trees is that the data needs to be prepared by making comparisons before it can be used to find the closest point to another point. Depending on how long it takes to prepare the data, it might actually be slower than other methods.


Though, most methods also needs to prepare their data as well, making it rather an argument about how the data is prepared and how long that takes. (Though, for really small datasets the brute force approach can be statistically the fastest.)

Another method of finding the closest point to another point or in a given direction of movement (as in ray tracing) is to group the points based on location. Quantizing it into smaller lists. Then we only need to check the points in the neighboring lists, or the lists relative to our direction. And quantizing data into lists is a relatively trivial task since it does no comparisons nor calculates any medians. This however has many downsides of its own, so it too won't be ideal in a lot of situations. (And one can recursively quantize the lists into new lists, how many items a list contains is though a debatable topic.)

I used Blender’s mathutils.

gaussian mixture with noise explanation is the best, ie, probability distribution function estimation

Many people know how hard it is to find G...

I love Computerphile. Thanks for educating the public! Have you considered a series on a confusing thing called Design Patterns by the Gang of Four?

If the root represents the starting point of your search, does it mean that one must create a K-d tree for every point so each point knows how to traverse on their own tree?

Locality sensitive hashing next!

Kraft Dinner trees?

The KD tree really shines for me when you represent it with a simple heap. You get an advanced data structure with range/radius searches, and nearest neighbor queries, all in logarithmic time for no extra memory! The kd tree would just be an array of points!

If you want to pick the split dimensions more smartly (like if you have a long, thin pointcloud) you only need one array of bytes that represents which split occurs at each node.

As an embedded engineer I like decision trees, It's all we can run on an MSP430. But you can do a lot with them if you know how to train them. Thanks WEKA!

Did he mention how to pick the point to split at? Because he's just picking optimals. If I have to search for the ones in the middles it's not actually worth it for dynamically moving items? Edit: Right. He picks the median point, he said.