Schmidhuber has a style I like: take a thing people usually leave as vibes, then ask what quantity would have to change inside an observer for the vibe to happen.
The key move in Driven by Compression Progress is not merely "beauty is compression." The sharper move is:
interestingness is the first derivative of subjective beauty.
Let an observer at time t have an internal compressor M_t. Given an object or
experience x, define its current subjective description length as:
$$ C_t(x) = L(x \mid M_t) $$
A toy definition of subjective beauty is:
$$ B_t(x) = -C_t(x) $$
The object feels beautiful when my current model gives it a short code. But it feels interesting when the code is actively getting shorter:
$$ I_t(x) = B_{t+1}(x) - B_t(x) $$
Since B_t(x) = -C_t(x), this is:
$$ I_t(x) = C_t(x) - C_{t+1}(x) $$
So an experience is interesting when it causes compression progress. Not maximum novelty, not maximum complexity, and not already-solved order. The interesting thing is the learnable pattern.
This is also why random noise is not interesting. A noisy stream can keep prediction error high forever, but it does not necessarily produce a better compressor. A better intrinsic reward is closer to:
$$ r_t^{\text{int}} = C_{\text{old}}(h_t) - C_{\text{new}}(h_t) $$
where both compressors are evaluated on the same history h_t.
The short version:
$$ \text{interestingness} = \frac{d}{dt} \text{subjective beauty} $$
The magic is not just that the world can be compressed. The magic is becoming able to compress it.