Mech Interp Day 0: Motivations

Starting my Mech Interp Journey

Change is never easy, and stepping away from astronomy after many years feels like giving up part of my identity. I don't regret my decision to pursue graduate school, those years have been nothing but fulfilling. It's a great privilege to dedicate time and effort to a pursuit of such intellectual purity. However, at the end of graduate school, one must once again cast thoughts toward the future. Remaining on the academic path brings uncertainty in many areas of life and often demands personal sacrifices: constant relocation, distance from loved ones, evolving responsibilities as teaching loads and funding pressures mount. Balancing these demands with other personal priorities is far from easy, and I found myself unable to envision navigating this path happily. I've also always been acutely aware that while astronomy is fascinating, there are infinite equally interesting areas of study and work—many of more immediate relevance to humanity. While changing trajectory has been difficult, deciding what to do next has been even more challenging. One area I'm strongly interested in pursuing is mechanistic interpretability.

The Rise of Neural Networks

Over the years, I've observed the explosion of machine learning, or more accurately, the rise of neural networks, deep learning, and large language models as they evolved from esoteric topics to universal adoption, taking root in all aspects of society. The social implications are tremendous, and it really does appear to be a watershed moment in how humans interact with technology. Yet so much remains mysterious about how these networks actually work. We train these systems on vast amounts of data, but their resultant capabilities have repeatedly exceeded expectations while theoretical understanding struggles to catch up. This reminds me of complex systems that display emergent behavior even with simple, local rule-based evolution. We know the full state of a trained neural network (every weight, bias, and computation that flows through it), but its overall capabilities still baffle us. Information is being processed in ways that seem opaque to us, so a field has crystallized around making sense of this opacity in ways we can understand and interpret. This is mechanistic interpretability (or "mech interp"), the science of understanding how machine learning networks learn to process information, much like neuroscience tries to understand how the biological brain does something similar. While young, this field is moving at lightning speed. I strongly believe that progress in mech interp is among the most important research being done today, given the reach and rate of growth of this technology. Preventing these systems from pursuing unintended goals (known as AI alignment) surely requires developing an understanding of how these networks do what they do.

A New Experience

While I've maintained interest in machine learning and neural networks over the years, my direct experience has been limited. The applications to my research were never convincing or promising enough (a lack of interpretability makes neural networks problematic for theoretical applications). Recently, however, I experimented with using Neural ODEs as a natural way of extending our usual process of modeling physical systems with differential equations through deep learning, while exploring symbolic regression to improve interpretability and generalizability. I also attended NeurIPS 2024 and got a feel for what the field was excited about, including in the context of scientific applications. It's been over half a year since then, and many of the big ideas such as MCP, multimodal inputs, and agentic AI have dominated advances in that time. Along the way, I've repeatedly encountered work being done on mechanistic interpretability, including papers from the Anthropic team. Beyond skimming these papers, I haven't devoted time to thinking more deeply about these ideas. Given that this is a field I'm interested in pursuing, I've decided to invest time in diving deeper into the ideas and research in the literature, and exploring where I might be able to contribute.

Why This Blog

Starting this research blog serves three purposes.

1. Mainly, I hope it will document my thoughts and ideas as they change and evolve while I learn and explore this new field. As I progress, it will be useful to return to earlier thoughts.
2. Second, I want to improve the clarity of my writing, since I often find it difficult to express thoughts without extensive refinement.
3. Finally, I'm hoping this imposes some level of self-accountability to keep at it regularly, since this will be on the side and doesn't overlap with my current work.