← All workshopsRead Distance and Similarity

Natural Language Processing

Compare direction without rewarding length

Equal lengths with perpendicular directions

Botie's query and document term vectors can have different lengths because a document repeats more terms. A direction comparison should remove that shared size effect.

BOTIE

The query and document vectors have the same length but occupy different coordinate directions. Should equal length imply a perfect match?
The inputs we are comparing

Query term vector

Example 1

  1. 01
  2. 12

Example 2

  1. 01
  2. 10

Document term vector

Example 1

  1. 02
  2. 14

Example 2

  1. 00
  2. 11
Result for example 1
1
Choose an explanation to test
A document covers only one of the query's directions

This check uses the pieces from “Compare term-vector directions using both lengths”.

BOTIE

Return cosine similarity for these nonzero term vectors. Keep both vector lengths in the calculation so that partial directional overlap is distinguished from a perfect match.
Query term vector
  1. 03
  2. 14
Document term vector
  1. 03
  2. 10

Use a number, text, or an array matching the requested result. For a matrix, put each row in brackets. Saving your prediction runs your current construction on these inputs and checks the earlier examples too.

Build a construction in the workspace above first. This case will test those pieces.

Why does the longer copy in the first case receive the same cosine as the original direction?

Choose a similarity measure according to which changes should leave the comparison unchanged.

This is a small construction problem. The lesson explains the assumptions and the parts of the full method that this example leaves out.

Work through the Python challenges →
Continue the courseLatent Semantic Analysis →
← A Vector for a Word