A Simple Cosine Similarity Example
In computer land everything is denoted as vectors. LLMs we use every day train on an unfathomable amount of data to determine which words are the most common "neighbors" of search topics and can use this to created a weighted model. All of this data is stored as vectors in a database. I will not be doing this. Sorry.
Creating our vectors
For simplicity let's start with a very small example. Let's pick some words and assign some common traits to some of my favorite video games.
Let's say we choose the traits is_fps, has_campaign, and is_strategy. Of course in the real world the are thousands
of traits, if not more. We'll explore that later.
Counter Strike = [1, 0, 0]Elden Ring = [0, 1, 0]Halo = [1, 1, 0]Stellaris = [0, 0, 1]Total War = [0, 1, 1]
Recall from the overview that alternate form for the dot product:
$$\mathbf a \cdot \mathbf b = \|\mathbf a\| \|\mathbf b\| \cos\theta$$
we can rewrite this to solve for $\cos\theta$: $$\cos\theta = \frac{\mathbf a \cdot \mathbf b}{\|\mathbf a\|\|\mathbf b\|}$$
We can then plug and play the values and see the similarities and differences. Let's look at Halo as our example. Halo is an FPS with a campaign mode. Let's compare this to Counter Strike, a team based FPS, and Stellaris, a Scifi strategy game. We see that between Halo and Counter Strike we get $\cos\theta = 1/\sqrt 2$. Between Halo and Stellaris however we see $\cos\theta = 0$. This shows us that Halo is more similar to Counter Strike than it is to Stellaris. The closer the cosine value is to 1, the more similar the options are. A value of 0 means that the options are completely unrelated (orthogonal), and a negative value means the two vectors are pointing in opposite directions.