Epistemic status: possibly trivial, but I hadn't heard it before.

TL;DR: What I thought of as a "flaw" in PCA—its inability to isolate pure metrics—might actually be a feature that aligns with our cognitive processes. We often think in terms of composite concepts (e.g., "Age + correlated attributes") rather than pure metrics, and this composite thinking might be more natural and efficient

Introduction

I recently found myself describing Principal Component Analysis (PCA) and pondering its potential drawbacks. However, upon further reflection, I'm reconsidering whether what I initially viewed as a limitation might actually be a feature. This led me to think about how our minds — and, potentially, language models — might naturally encode information using correlated attributes.

An important aspect of this idea is the potential conflation between the metric we use to measure something and the actual concept we're thinking about. For instance, when we think about [...]

---

Outline:

(00:35) Introduction

(01:51) 1 - Age and Associated Attributes in Children

(03:52) 2 - Price and Quality of Goods

(04:38) 3 - Size and Weight of Objects

(05:30) Implications for Information Encoding

(06:27) Rethinking PCA and Dimensional Reduction

---

First published:
July 23rd, 2024

Source:
https://www.lesswrong.com/posts/EJFvitiQXmLHXgtom/confusing-the-metric-for-the-meaning-perhaps-correlated

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Confusing the metric for the meaning: Perhaps correlated attributes are ‘natural’” by NickyP

LessWrong (30+ Karma)