Wisdom of My Crowd - The Nerdy Part
This is the technical part: math, information, computatioin, physics, and of course programming and fun !
Math
Categogry Thoery
-
Category theory was born out of the quest to make precise the idea that some transformations are "natural" and some aren't.
-
I go back and forth between thinking category theory is a powerful secret weapon and thinking it’s a waste of time. Can be both.
Math is lossy
Math is a concise way to represent how a system might behave. It is an anchor with which to fasten rational thought. But simplification is never free. Math is not a lossless compression of nature’s information.
Causality
-
"When people talk about interpretability, they are usually talking about causality, which I think is important and more people should work on it"-Yann Lecun #NIPS2017
-
Would Pearl's P(Y | do(X = x)) notation be a little easier to understand if Pearl also wrote P(Y | X = x) as P(Y | observe(X = x))?
-
I wish we'd replace the "prediction" vs "explanation" dichotomy with Meehl's "prediction w/o intervention" vs "prediction w/ intervention".
Physics
Natural math is physics
Physics is when nature checks your math.
Quantum
Someone asked: What are the most important ideas in physics over the past 30 years? Three immediately came to mind: The holographic principle, topological order, quantum error correction. What's lovely is that these three ideas are so closely related.
Lorentzian
The weirdest thing yet discovered by physics is not that the world is 4-dimensional but that its geometry is Lorentzian. That it is curved is prosaic by comparison. So are Entanglement and the multiverse.
Real Sciences
There is a core of what is called "science" (physics, chemistry, almost all of biology and computer science) that is quite sound. Some of the rest of "science" is sound. Most of the rest is quite pathological.
Programming
Coding
I feel like programming is ~60% developing the right mental model, 40% debugging and testing, and less than 1% writing code. At least when you're solving a novel problem.
Math and programming
mathematics is beauty, programming is utility, and their intersection is both at once
Break the leaky/lousy abstractions
so much alpha in actually understanding how things work the abstraction is begging to be broken break it
FP vs OO
How did they fool us into object-oriented programming?
I can't imagine anything more joyful and yet scalable as functions and passing data between them, organized by simple packages.
Functional Programming and Design
As a corollary to my previous thread… Career pro tip: if you want to get good, REALLY good, at designing complex and stateful distributed systems at scale in real-world environments, learn functional programming. It is an almost perfectly identical skillset.
FP with ChatGPT
It turns out the chatGPT is really good at defining applicative and monad instances for Haskell types, even for examples where it's difficult to do this by hand, e.g. involving continuations. It's a great time saver!
Lambda calculs
Mark my words: there is no way gradient descent finds programs faster than well-pruned, optimally shared search in the space of dependently-typed lambda terms.
Abstraction
When programmers tired of writing tedious Assembly code, they came up with higher-level languages like C to generate Assembly for them.
But somehow in 2024, the tech industry uses AI to generate boilerplate code instead of creating more expressive languages 😞
FP vs OO
Once you’ve adapted the FP mindset, I think objects become even more complex. It’s like passing around tiny VMs, all with their own functions and internal state, so many side-effects, everywhere! That people think it’s “simpler” than FP boggles the mind.
No jargons
To make FP mainstream, next time you create a new functional language, instead of using terminology
⚗️ Type classes, Algebraic Data Types, and Monads
make yourself (and the community) a favour and call them
🐹 Interfaces, Choices and Workflows
Compsitionality, not compasability
My take on the whole “pure FP” current thing is that I don’t super care about purity per se. For me it’s only a vehicle to compositionality, which is what I’m really after.
There are only two ways to understand programs:
-
You can understand it all at once, where none of the parts mean anything without all the others.
-
You can understand it compositionally, where your understanding of the whole is composed of your understanding of the parts.
Side effects are hard barriers to composition, so to the extent you have them your program is of the former kind.
I vastly prefer the latter kind because it makes it easier and more pleasant for me to understand, change, and test such code.
Abstraction and infrastructure
Two ways of sharing:
easy to underappreciate the value of good abstractions and reliable infrastructure
Software Engineer
If you can't build an entire app using just
🔰 Recursion 🔰 Immutable data 🔰 Records 🔰 Algebraic Data Types 🔰 Pattern Matching 🔰 Higher-Order Functions 🔰 Parametric Polymorphism
Are you even a Software Engineer?
Fun
You can not learn Haskell ?
The Go Programming Language literally exists because Google doesn't think you're smart enough to learn Haskell.
Hackers
i'm beginning to learn that there is a class of people that are just super neets. they are so proficient at anything that they do that they make money almost by accident. they don't value anything other than having fun. gwern is an example
Information Theory (just for fun, not true)
Information theory is essentially probability on a log scale.
Open Source
Open source means we make the plans available so anyone can build the house and improve on them. Actually building a good house is valuable and a service people pay for.
Computer Science
Computer science is the bastard child of mathematics and electrical engineering.
Physicist and stats
Asking a physicist to sit through a statistics lecture is like inviting an Italian over for dinner and serving frozen pizza.
Who is the bastard ?
Life tip: When you sign up for anything online, put the website’s name as your middle name. That way when you receive spam/advert emails, you will know who sold your info.