"It is geometry, applied to meaning, made possible by training models that learned the regularities of language by reading a great deal of it." Love this line. A great deal, indeed. While not understanding a single word. hehe
Hi Rob, there are two concepts here I could help clarify:
1. Starting from the model “reads a great deal,” it is a bit like how a student gradually learns how paragraphs work: recurring patterns, grammar, linguistic structure, and the relationships between ideas. In that sense, the model is learning patterns of meaning from patterns in language.
2. However, computer machines cannot technically "read". It processes text by turning tokens into numerical representations, a process we call vectorization. Vectorization turns words, phrases, and contexts into lists of numbers. Those numbers are not “meaning” in the human sense, but they give the machine a way to represent relationships between words and concepts. It's like me using a dictionary to look up words and their meanings; a computer uses those numbers to check its own words and meanings - not through a traditional dictionary, but through relationships among the numbers -> geometry.
"It is geometry, applied to meaning, made possible by training models that learned the regularities of language by reading a great deal of it." Love this line. A great deal, indeed. While not understanding a single word. hehe
Hi Rob, there are two concepts here I could help clarify:
1. Starting from the model “reads a great deal,” it is a bit like how a student gradually learns how paragraphs work: recurring patterns, grammar, linguistic structure, and the relationships between ideas. In that sense, the model is learning patterns of meaning from patterns in language.
2. However, computer machines cannot technically "read". It processes text by turning tokens into numerical representations, a process we call vectorization. Vectorization turns words, phrases, and contexts into lists of numbers. Those numbers are not “meaning” in the human sense, but they give the machine a way to represent relationships between words and concepts. It's like me using a dictionary to look up words and their meanings; a computer uses those numbers to check its own words and meanings - not through a traditional dictionary, but through relationships among the numbers -> geometry.