AF - Natural abstractions are observer-dependent: a conversation with John Wentworth by Martín Soto

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Natural abstractions are observer-dependent: a conversation with John Wentworth, published by Martín Soto on February 12, 2024 on The AI Alignment Forum.
The conversation
Martín:
Any write-up on the thing that "natural abstractions depend on your goals"? As in, pressure is a useful abstraction because we care about / were trained on a certain kind of macroscopic patterns (because we ourselves are such macroscopic patterns), but if you cared about "this exact particle's position", it wouldn't be.[1]
John:
Nope, no writeup on that.
And in the case of pressure, it would still be natural even if you care about the exact position of one particular particle at a later time, and are trying to predict that from data on the same gas at an earlier time. The usual high level variables (e.g.
pressure, temperature, volume) are summary stats (to very good approximation) between earlier and later states (not too close together in time), and the position of one particular particle is a component of that state, so (pressure, temperature, volume) are still summary stats for that problem.
The main loophole there is that if e.g. you're interested in the 10th-most-significant bit of the position of a particular particle, then you just can't predict it any better than the prior, so the empty set is a summary stat and you don't care about any abstractions at all.
Martín:
Wait, I don't buy that:
The gas could be in many possible microstates. Pressure partitions them into macrostates in a certain particular way. That is, every possible numerical value for pressure is a different macrostate, that could be instantiated by many different microstates (where the particle in question is in very different positions).
Say instead of caring about this partition, you care about the macrostate partition which tracks where that one particle is.
It seems like these two partitions are orthogonal, meaning that conditioning on a pressure level gives you no information about where the particle is (because the system is symmetric with respect to all particles, or something like that).
[This could be false due to small effects like "higher pressure makes it less likely all particles are near the center" or whatever, but I don't think that's what we're talking about. Ignore them for now, or assume I care about a partition which is truly orthogonal to pressure level.]
So tracking the pressure level partition won't help you.
It's still true that "the position of one particular particle is a component of the (micro)state", but we're discussing which macrostates to track, and pressure is only a summary stat for some macrostates (variables), but not others.
John:
Roughly speaking, you don't get to pick which macrostates to track. There are things you're able to observe, and those observations determine what you're able to distinguish.
You do have degrees of freedom in what additional information to throw away from your observations, but for something like pressure, the observations (and specifically the fact that observations are not infinite-precision) already pick out (P, V, T) as the summary stats; the only remaining degree of freedom is to throw away even more information than that.
Applied to the one-particle example in particular: because of chaos, you can't predict where the one particle will be (any better than P, V, T would) at a significantly later time without extremely-high-precision observations of the particle states at an earlier time.
Martín:
Okay, so I have a limited number of macrostate partitions I can track (because of how my sensory receptors are arranged), call this set S, and my only choice is which information from that to throw out (due to computational constraints), and still be left with approximately good models of the environment.
(P, V, T) is considered a natural abstraction in this sit...