The more mistakes you make, the more you learn. Let them make mistakes.
See also the prompt engineering document.
Ilya Sutskever presents an Observation on Generalization https://www.youtube.com/watch?v=AKMuA_TVz3A&t=4s
“Why would data have regularities that our machine learning models can capture?”
Perplexity explaining an analogy between homography and the way transformers simulate alternative architecture internally https://www.perplexity.ai/search/what-is-sgd-in-this-context-yo-rLzyIfSWRE27gHO4eut_WA#1
“Ilya Sutskever views deep neural networks, including transformers, as universal parallel computers capable of universal computation given enough scale. These models execute sequential operations (like attention layers) in parallel across massive hardware, simulating arbitrary programs during training via optimization like Stochastic Gradient Descent (SGD).
Homography refers to a projective transformation in computer vision and geometry that maps points from one plane (or image) to another while preserving lines and incidence relations, typically via a 3x3 matrix. It’s like a “micro” version because neural networks, especially transformers, can efficiently simulate one architecture inside another through such transformations, much as computers emulate each other.
The “micro homography” idea emphasizes robustness: just as a homography seamlessly warps one planar view into another without losing structural fidelity (e.g., in image stitching), transformers can “warp” or simulate alternative architectures internally. This makes them interchangeable in capability—depth vs. width, dense vs. sparse attention—when optimization is strong, as minor changes are absorbed like viewpoint shifts in projective geometry. Architectural tweaks matter mainly for bottlenecks or efficiency, not core expressivity.
https://www.perplexity.ai/search/what-does-sergey-s-return-mean-jVvbrHr2T3Ck51YQu4jOSQ#2
“Yann LeCun, argues that AGI often defined as human-level or superhuman intelligence across all domains is a flawed, overhyped concept because intelligence is inherently relative and context-specific. Humans excel in narrow, evolved skills like social reasoning, tool use, and survival in physical environments, but lag in raw pattern recognition or data processing compared to animals or machines; for instance, cats outperform humans in visuomotor tasks.”
“Why It Still Matters”
“LeCun’s view doesn’t negate practical AI progress; it reframes”AGI” as unnecessary hype, emphasizing “human-level AI” for specific, valuable capabilities like world modeling or robotics, where Google (under Brin’s push) and others compete. In the poster’s context, Brin’s return signals Google’s execution on these frontiers—Gemini advancements in multimodal models—not chasing an ill-defined AGI finish line. Real-world dominance in video/images (YouTube data) and chips (TPUs) drives usage wins, regardless of terminology.”