
## Starting my Mech Interp Journey

Change is never easy, and stepping away from astronomy after many years feels like giving up part of my identity. I don't regret my decision to pursue graduate school, those years have been nothing but fulfilling. It's a great privilege to dedicate time and effort to a pursuit of such intellectual purity. However, at the end of graduate school, one must once again cast thoughts toward the future.
Remaining on the academic path brings uncertainty in many areas of life and often demands personal sacrifices: constant relocation, distance from loved ones, evolving responsibilities as teaching loads and funding pressures mount. Balancing these demands with other personal priorities is far from easy, and I found myself unable to envision navigating this path happily. I've also always been acutely aware that while astronomy is fascinating, there are infinite equally interesting areas of study and work—many of more immediate relevance to humanity.
While changing trajectory has been difficult, deciding what to do next has been even more challenging. One area I'm strongly interested in pursuing is mechanistic interpretability.

## The Rise of Neural Networks

Over the years, I've observed the explosion of machine learning, or more accurately, the rise of neural networks, deep learning, and large language models as they evolved from esoteric topics to universal adoption, taking root in all aspects of society. The social implications are tremendous, and it really does appear to be a watershed moment in how humans interact with technology.
Yet so much remains mysterious about how these networks actually work. We train these systems on vast amounts of data, but their resultant capabilities have repeatedly exceeded expectations while theoretical understanding struggles to catch up. This reminds me of complex systems that display emergent behavior even with simple, local rule-based evolution. We know the full state of a trained neural network (every weight, bias, and computation that flows through it), but its overall capabilities still baffle us.
Information is being processed in ways that seem opaque to us, so a field has crystallized around making sense of this opacity in ways we can understand and interpret. This is mechanistic interpretability (or "mech interp"), the science of understanding how machine learning networks learn to process information, much like neuroscience tries to understand how the biological brain does something similar.
While young, this field is moving at lightning speed. I strongly believe that progress in mech interp is among the most important research being done today, given the reach and rate of growth of this technology. Preventing these systems from pursuing unintended goals (known as AI alignment) surely requires developing an understanding of how these networks do what they do.

## A New Experience

While I've maintained interest in machine learning and neural networks over the years, my direct experience has been limited. The applications to my research were never convincing or promising enough (a lack of interpretability makes neural networks problematic for theoretical applications).
Recently, however, I experimented with using Neural ODEs as a natural way of extending our usual process of modeling physical systems with differential equations through deep learning, while exploring symbolic regression to improve interpretability and generalizability. I also attended NeurIPS 2024 and got a feel for what the field was excited about, including in the context of scientific applications.
It's been over half a year since then, and many of the big ideas such as MCP, multimodal inputs, and agentic AI have dominated advances in that time. Along the way, I've repeatedly encountered work being done on mechanistic interpretability, including papers from the Anthropic team. Beyond skimming these papers, I haven't devoted time to thinking more deeply about these ideas.
Given that this is a field I'm interested in pursuing, I've decided to invest time in diving deeper into the ideas and research in the literature, and exploring where I might be able to contribute.

## Why This Blog

Starting this research blog serves three purposes.
1. Mainly, I hope it will document my thoughts and ideas as they change and evolve while I learn and explore this new field. As I progress, it will be useful to return to earlier thoughts.
2. Second, I want to improve the clarity of my writing, since I often find it difficult to express thoughts without extensive refinement.
3. Finally, I'm hoping this imposes some level of self-accountability to keep at it regularly, since this will be on the side and doesn't overlap with my current work.

Mech Interp Day 0: Motivations


The previous iteration of this website featured an interactive 3D pulsating sphere built using Three.js. For the sake of a cleaner feel, I decided to remove it for this iteration, but found the experience rewarding. The animation I originally had can be found at the botto of this post! The capability for 3D graphics that Three.js provide is rich in potential, especially since modern web development has evolved far beyond static pages. Today's users expect rich, interactive experiences that feel more like native applications than traditional websites.

## The Technology Stack

Combining several powerful technologies can create truly engaging web experiences. The combination I initially went with included:

- **Three.js** for 3D graphics and WebGL rendering
- **Framer Motion** for smooth animations and transitions
- **React** for component-based architecture
- **Next.js** for performance optimization

## Why 3D on the Web?

Three.js has revolutionized how we think about web interfaces. Here's a simple example of creating a rotating cube:

```javascript
import * as THREE from 'three';

const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
const renderer = new THREE.WebGLRenderer();

const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
const cube = new THREE.Mesh(geometry, material);

scene.add(cube);
camera.position.z = 5;

function animate() {
    requestAnimationFrame(animate);
    cube.rotation.x += 0.01;
    cube.rotation.y += 0.01;
    renderer.render(scene, camera);
}

animate();
```

## Animation with Framer Motion

Framer Motion makes it incredibly easy to add smooth animations to React components:

```jsx
import { motion } from 'framer-motion';

const AnimatedCard = () => {
  return (
    
      Interactive Card
      This card animates on load and hover!
    
  );
};
```

## My Experience

The combination of Three.js and Framer Motion in a React/Next.js environment provides an incredibly powerful toolkit for creating memorable web experiences. The key is finding the right balance between visual impact and performance. The rest of this post will go through how I set up my initial animation, with the final product at the end!

## Uniform Sampling of Points on A Sphere

The key to creating a convincing sphere lies in proper point distribution. Unlike naive approaches that create clustering at poles, we use **spherical coordinates** with uniform random sampling:

```javascript
const points = [];
for (var i = 0; i < 1500; i++) {
  var vertex = new THREE.Vector3();

  // Uniform random sampling on sphere surface
  var u = THREE.MathUtils.randFloat(0, 1);
  var v = THREE.MathUtils.randFloat(0, 1);
  var theta = 2 * Math.PI * u;           // Azimuthal angle
  var phi = Math.acos(2 * v - 1);        // Polar angle

  // Convert to Cartesian coordinates
  vertex.x = 3.5 * Math.sin(phi) * Math.cos(theta);
  vertex.y = 3.5 * Math.sin(phi) * Math.sin(theta);
  vertex.z = 3.5 * Math.cos(phi);

  points.push(vertex);
}
```

This approach ensures **uniform distribution** across the sphere surface, avoiding the visual artifacts of simpler random placement methods.

## GLSL Shader

The pulsating light effect is achieved through custom **vertex and fragment shaders** that run directly on the GPU:

### Shader Uniforms Setup

```javascript
const shaderPoint = THREE.ShaderLib.points;
const uniforms = THREE.UniformsUtils.clone(shaderPoint.uniforms);
uniforms.time = { value: 0 };
uniforms.color = { type: "v3", value: colorValue };

const pMaterial = new THREE.ShaderMaterial({
  uniforms,
  transparent: true,
  depthWrite: false,
  blending: THREE.AdditiveBlending,  // Creates light emission effect
  vertexShader,
  fragmentShader,
});
```

### The Shader Pipeline

This particle system consists of two interconnected shaders:

- **Vertex Shader** ("vert.glsl") - Handles particle positioning, sizing, and movement
- **Fragment Shader** ("frag.glsl") - Controls particle appearance, color, and glow effects

### 3D Simplex Noise Implementation

The vertex shader begins with a complete **Simplex noise** implementation - a sophisticated algorithm for generating natural-looking randomness:

```glsl
float snoise(vec3 v) {
    // 85 lines of math
    // Creates smooth, continuous 3D noise
    return 42.0 * dot(m*m, vec4(dot(p0,x0), dot(p1,x1), dot(p2,x2), dot(p3,x3)));
}
```

**Why Simplex Noise?** Unlike basic random functions, Simplex noise provides:
- **Continuous gradients** - no sudden jumps or artifacts
- **3D coherence** - neighboring points have similar values
- **Performance optimization** - faster than Perlin noise
- **Natural patterns** - mimics organic movement and growth

### Organic Particle Movement

The main vertex shader transforms each particle's position using time-based trigonometric functions:

```glsl
vec3 newPos = position;

newPos.x += sin(time + position.x * position.y) * 0.08;
newPos.y += cos(time + position.x * position.y * 1.1) * 0.08;
newPos.z += cos(time + position.x * position.y * 1.3) * 0.08;
```

**Mathematical Breakdown:**
- **Base oscillation**: "sin(time)" and "cos(time)" create rhythmic movement
- **Position coupling**: "position.x * position.y" makes each particle's movement unique
- **Frequency variation**: Multipliers prevent synchronized motion
- **Amplitude control**: "0.08" keeps movement subtle and elegant

This creates a **Lissajous-like pattern** where each particle follows its own complex orbital path, determined by its starting position.

### Dynamic Particle Sizing

The most sophisticated aspect is the noise-driven size variation:

```glsl
gl_PointSize = 50. + snoise(position * 0.05 + vec3(0, 0, vtime * 0.1)) * 50.;
gl_PointSize *= 0.5;
```

**Size Calculation Explained:**
- **Base size**: "50." pixels provides consistent minimum visibility
- **Noise sampling**: "snoise(position * 0.05 + ...)" creates spatial variation
- **Temporal drift**: "vec3(0, 0, vtime * 0.1)" makes noise evolve over time
- **Amplitude**: "* 50." allows sizes to range from 0 to 100 pixels
- **Final scaling**: "* 0.5" reduces overall scale to 0-50 pixels

The result is particles that **breathe** - growing and shrinking organically as the noise field evolves through time.

### Time-Based Animation

The pulsating effect is driven by a time uniform that updates every frame:

```javascript
const animate = (time) => {
  // Update shader time for pulsating effect
  pMaterial.uniforms.time.value = time * 0.004;
  
  // Continue animation loop
  requestAnimationFrame(animate);
  renderer.render(scene, camera);
}
```

The **AdditiveBlending** mode creates the characteristic light emission, making particles appear to glow and blend naturally when they overlap.

## Responsive Color System

The particle system adapts to the user's color mode preference through dynamic uniform updates:

```javascript
// Real-time color mode detection and adaptation
if (localStorage.getItem("chakra-ui-color-mode") === "dark") {
  pMaterial.uniforms.color.value = new THREE.Color(0xffffff);  // White particles
} else {
  pMaterial.uniforms.color.value = new THREE.Color(0x000000);  // Black particles
}
```

This creates a seamless experience where the 3D scene automatically adapts to the user's interface preferences without requiring page refreshes.

## Smooth Camera Animations

The initial camera movement uses an **easing function** to create natural motion:

```javascript
function easeOutCirc(x) {
  return Math.sqrt(1 - Math.pow(x - 1, 4));
}

// Camera animation during first 100 frames
if (frame <= 100) {
  const rotSpeed = -easeOutCirc(frame / 120) * Math.PI * 20;
  
  camera.position.x = p.x * Math.cos(rotSpeed) + p.z * Math.sin(rotSpeed);
  camera.position.z = p.z * Math.cos(rotSpeed) - p.x * Math.sin(rotSpeed);
  camera.lookAt(target);
} else {
  // Switch to user-controlled orbit after animation
  controls.update();
}
```

After the initial animation completes, control transitions to **OrbitControls** for user interaction, with automatic rotation enabled.

## Performance Optimizations

### GPU-Accelerated Rendering
- **ShaderMaterial** for GPU-based calculations

### Efficient Animation Loop
```javascript
let req = null;
const animate = (time) => {
  req = requestAnimationFrame(animate);
  
  // Minimal CPU calculations
  // GPU handles particle transformations
  
  renderer.render(scene, camera);
}

// Proper cleanup
return () => {
  cancelAnimationFrame(req);
  renderer.domElement.remove();
  renderer.dispose();
}
```

## Final Product
Here is the final animation, best viewed in dark mode!
<hr />
<voxel-art />

Building Interactive Web Experiences


This document summarizes the complete implementation of a markdown-based blog system using Next.js, Chakra UI, react-markdown, and KaTeX for mathematical equations.

## Tech Stack

- **Next.js** - React framework with static site generation
- **Chakra UI** - Component library for styling
- **react-markdown** - Markdown to React component converter
- **KaTeX** - LaTeX equation rendering
- **gray-matter** - Frontmatter parsing
- **Framer Motion** - Animations

## Required Dependencies

```bash
npm install react-markdown remark-gfm rehype-highlight rehype-raw gray-matter katex rehype-katex remark-math
```

## File Structure

```
project/
├── components/
│   └── BlogPost.jsx
├── lib/
│   └── posts.js
├── pages/
│   ├── writing.js (blog index)
│   └── blog/
│       └── [slug].js (individual posts)
├── posts/
│   ├── my-first-post.md
│   └── second-blog-post.md
└── pages/_app.js
```

## Core Components

### lib/posts.js - File System Functions
```javascript
import fs from 'fs';
import path from 'path';
import matter from 'gray-matter';

const postsDirectory = path.join(process.cwd(), 'posts');

export function getAllPosts() {
  const fileNames = fs.readdirSync(postsDirectory);
  const allPostsData = fileNames.map((fileName) => {
    const slug = fileName.replace(/\.md$/, '');
    const fullPath = path.join(postsDirectory, fileName);
    const fileContents = fs.readFileSync(fullPath, 'utf8');
    const { data, content } = matter(fileContents);

    return {
      slug,
      content,
      ...data,
    };
  });

  return allPostsData.sort((a, b) => (a.date < b.date ? 1 : -1));
}

export function getPostBySlug(slug) {
  const fullPath = path.join(postsDirectory, `${slug}.md`);
  const fileContents = fs.readFileSync(fullPath, 'utf8');
  const { data, content } = matter(fileContents);

  return {
    slug,
    content,
    ...data,
  };
}
```

### components/BlogPost.jsx - Markdown Renderer
```javascript
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import remarkMath from 'remark-math';
import rehypeHighlight from 'rehype-highlight';
import rehypeKatex from 'rehype-katex';
import { 
  Box, 
  Heading, 
  Text, 
  Code, 
  Divider, 
  Link,
  UnorderedList,
  OrderedList,
  ListItem,
  useColorModeValue
} from '@chakra-ui/react';

const BlogPost = ({ content }) => {
  const blockquoteBg = useColorModeValue('gray.50', 'gray.700');
  const inlineCodeBg = useColorModeValue('gray.100', 'gray.600');
  const inlineCodeColor = useColorModeValue('gray.800', 'gray.100');

  return (
    <Box maxW="800px" mx="auto" p={6}>
      <ReactMarkdown
        remarkPlugins={[remarkGfm, remarkMath]}
        rehypePlugins={[rehypeHighlight, rehypeKatex]}
        components={{
          h1: ({ children }) => (
            <Heading as="h1" size="2xl" mb={6} mt={8}>
              {children}
            </Heading>
          ),
          h2: ({ children }) => (
            <Heading as="h2" size="xl" mb={4} mt={6}>
              {children}
            </Heading>
          ),
          h3: ({ children }) => (
            <Heading as="h3" size="lg" mb={3} mt={5}>
              {children}
            </Heading>
          ),
          p: ({ children }) => (
            <Text mb={4} lineHeight="1.7">
              {children}
            </Text>
          ),
          ul: ({ children }) => (
            <UnorderedList mb={4} spacing={2}>
              {children}
            </UnorderedList>
          ),
          ol: ({ children }) => (
            <OrderedList mb={4} spacing={2}>
              {children}
            </OrderedList>
          ),
          li: ({ children }) => (
            <ListItem>{children}</ListItem>
          ),
          code: ({ inline, children, className }) => {
            return inline ? (
              <Code 
                px={2} 
                py={1} 
                bg={inlineCodeBg}
                color={inlineCodeColor}
                borderRadius="md"
                suppressHydrationWarning={true}
              >
                {children}
              </Code>
            ) : (
              <Box mb={4}>
                <Code
                  as="pre"
                  display="block"
                  p={4}
                  bg="gray.900"
                  color="white"
                  _dark={{ bg: "gray.800" }}
                  borderRadius="md"
                  overflow="auto"
                  suppressHydrationWarning={true}
                >
                  {children}
                </Code>
              </Box>
            );
          },
          a: ({ href, children }) => (
            <Link href={href} color="blue.500" isExternal>
              {children}
            </Link>
          ),
          blockquote: ({ children }) => (
            <Box
              as="blockquote"
              borderLeft="4px solid"
              borderColor="blue.500"
              pl={4}
              py={2}
              mb={4}
              fontStyle="italic"
              bg={blockquoteBg}
              _dark={{ bg: "gray.700", borderColor: "blue.300" }}
              borderRadius="md"
              suppressHydrationWarning={true}
            >
              {children}
            </Box>
          ),
          hr: () => <Divider my={6} />,
        }}
      >
        {content}
      </ReactMarkdown>
    </Box>
  );
};

export default BlogPost;
```

### pages/writing.js - Blog Index
```javascript
import NextLink from 'next/link'
import { Box, Container, Heading, Link, Text } from '@chakra-ui/react'
import Layout from '../components/layouts/article'
import Section from '../components/section'
import { getAllPosts } from '../lib/posts'

const Writing = ({ posts = [] }) => {
  return (
    <Layout title="Posts">
      <Container>
        <Heading as="h3" fontSize={20} mb={4} mt={5}>
          Posts
        </Heading>

        {posts && posts.map((post, index) => (
          <Section key={post.slug} delay={0.1 + (index + 1) * 0.1}>
            <Box my={4}>
              <Text fontSize="sm" color="gray.500" display="inline" mr={3}>
                {new Date(post.date + 'T00:00:00').toLocaleDateString('en-US', {
                  year: 'numeric',
                  month: 'short',
                  day: 'numeric',
                  timeZone: 'UTC'
                })}
              </Text>
              <Link as={NextLink} href={`/blog/${post.slug}`}>
                {post.title}
              </Link>
            </Box>
          </Section>
        ))}

      </Container>
    </Layout>
  )
}

export async function getStaticProps() {
  try {
    const posts = getAllPosts();
    return {
      props: {
        posts,
      },
    };
  } catch (error) {
    console.error('Error in getStaticProps:', error);
    return {
      props: {
        posts: [],
      },
    };
  }
}

export default Writing
```

### pages/blog/[slug].js - Individual Post Pages
```javascript
import { getAllPosts, getPostBySlug } from '../../lib/posts';
import BlogPost from '../../components/BlogPost';
import { Box, Heading, Text } from '@chakra-ui/react';

export default function Post({ post }) {
  return (
    <Box>
      <Box textAlign="center" mb={8}>
        <Heading as="h1" size="3xl" mb={4}>
          {post.title}
        </Heading>
        <Text color="gray.600">{post.date}</Text>
      </Box>
      <BlogPost content={post.content} />
    </Box>
  );
}

export async function getStaticProps({ params }) {
  const post = getPostBySlug(params.slug);
  return {
    props: {
      post,
    },
  };
}

export async function getStaticPaths() {
  const posts = getAllPosts();
  const paths = posts.map((post) => ({
    params: { slug: post.slug },
  }));

  return {
    paths,
    fallback: false,
  };
}
```

## Markdown Features Supported

### Basic Formatting
- **Bold text** and *italic text*
- Headers (H1, H2, H3)
- Paragraphs with proper spacing
- Horizontal rules
- Links (internal and external)

### Lists
- Unordered lists with bullet points
- Ordered lists with numbers
- Proper spacing between items

### Code
- Multi-line code blocks with language-specific highlighting
- Dark/light mode responsive styling

### Blockquotes
> Styled blockquotes with left border. Dark/light mode responsive

### Mathematical Equations
- Inline math: $E = mc^2$
- Block equations: $$\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}$$
- Complex expressions with proper LaTeX rendering

## Frontmatter Structure
```yaml
---
title: "Post Title"
date: "2025-01-15"
excerpt: "Brief description of the post"
tags: ["tag1", "tag2", "tag3"]
---
```

## Development Workflow

1. Create markdown files in posts/ directory
2. Posts automatically appear on /writing page

## Troubleshooting Notes

- **Date Issues**: Use `new Date(post.date + 'T00:00:00')` to prevent timezone shifts
- Inline equations not working

Blog Implementation Summary


## Welcome to My Blog

This is my first blog post! I will just test functionality in this post, and write up how I implemented post functionality in a future post.

## Code Example

```python
def hello_world():
    print("Hello, World!")
    return True
``` 

## More Examples

> This is a blockquote with some important information.

Check out [this link](https://example.com) for more info.

Here's some **bold text** and *italic text*.

### Lists

Unordered list:
- Item 1
- Item 2
- Item 3

Ordered list:
1. First item
2. Second item
3. Third item 

### Equations
This is an inline equation: $E = mc^2$.

In web animations, we often use trigonometric functions. The general form of a sine wave is:

$$
y(x,t) = A \sin(kx - \omega t + \phi)
$$

Where:
- $A$ is the amplitude
- $k$ is the wave number  
- $\omega$ is the angular frequency
- $\phi$ is the phase shift

This equation is perfect for creating smooth, natural animations in Three.js!

That's all for now!
