This is a running list of problems I keep thinking about across machine intelligence, computing, mathematics, and science:
Compute: How to maximize intelligence per unit of input resource?
- What is the minimum physical work required to generate one useful token?
- What is the minimum mathematical work required to generate one useful token?
- How much of inference is fundamentally necessary versus an artifact of today's transformer implementation?
- Where does energy actually go when producing a token?
- What is the minimum achievable joules/token?
- Are FLOPs even the right measure? Or should we care much more about bytes moved / tokens per joule?
- Could we get another 100x efficiency improvement without sacrificing model capability?
Memory movement: Is moving data the real enemy?
- Why are we moving trillions of bits around just to perform relatively cheap arithmetic on them?
- Will memory movement continue to remain the dominant cost in AI systems?
- What are all the ways to avoid moving weights around?
- How far can sparsity reduce memory reads without degrading model performance? If so, on which models/ datasets/workloads?
- On the memory axis, what's going to fundamentally change the scaling curve? Is it compute-in-memory, near-memory compute, caching, quantization, sparsity, recomputation, locality, something else?
Chips: Do we need a different physical substrate?
- What would AI hardware look like if designed from first principles around energy rather than FLOPs?
- Are GPUs the right machine for intelligence or just the machine we happened to have?
- What type of physical substrates can deliver energy savings? Will it require completely new models and training procedures?
- Can existing foundation models be mapped onto radically different hardware or should models be co-designed with the chip?
- Is there a genuine "hardware lottery"? Do algorithms succeed partly because available hardware happens to favor them?
Models: How much of today's architecture is fundamental?
- What does "same intelligence with less computation" actually mean mathematically?
- What is the irreducible algorithm for intelligence vs baggage inherited from how LLMs evolved?
- Can you separate model efficiency from compute efficiency or are they inherently coupled?
- Why transformers?
- What parts of transformer computation can be removed?
- What exactly has to happen for a model to produce intelligence?
- Does an alternative architecture need to train its own foundation model from scratch?
- How much can we sparsify? What happens if only a tiny fraction of neurons fire?
Memory + Learning: Where should knowledge live?
- Is memory becoming a distinct layer alongside the foundation model and agent/coding harness?
- Is continual learning just another form of post-training or something fundamentally different?
- Should an intelligent system's knowledge be encoded in its weights at all?
- When should information live in context vs live in external memory vs consolidated into weights?
- How can a system continually update knowledge without catastrophic forgetting?
- Can external memory remain structured and attributable even when the underlying model changes?
Reasoning: What actually constitutes machine intelligence?
- Does scaling next-token prediction genuinely create reasoning?
- Why are language models so good at coding and formal symbolic tasks?
- Where does language-only learning break down?
- Do spatial, temporal, causal, and physical reasoning require fundamentally different representations?
- How do we distinguish reasoning from sophisticated pattern completion?
- What infra is required for verifiable reasoning?
- Can agents become reliable through verifiers + evaluators + execution environments + search + control loops even if the underlying model remains imperfect?
World models: Does intelligence need to simulate reality?
- What's the difference between intelligence that describes the world and intelligence that can predict/interact with it?
- Are world models a major architecture beyond language models?
- How should machines represent dynamics rather than sequences of tokens?
- Why neural operators? Are they important because they learn mappings between entire functions/physical fields rather than finite-dimensional vectors?
- Could physical-world models become foundational models for science and engineering?
- Is simulation + RL the route from passive models to systems that learn, reason, and act?
AI systems: Where does the model end?
- Will neural network continue to remain the unit of intelligence?
- Is the real product the model or the model + memory + tools + runtime + verifier + environment?
- What belongs in the weights versus the surrounding system?
- How should inference-time compute be allocated?
- Can additional search/computation at inference substitute for larger models?
- What infra will long-running autonomous agents require?
- What is the equivalent of an operating system for machine intelligence?
Power: Does AI ultimately become an energy problem?
- What do "machine intelligence plants" look like?
- What is the smallest economically viable machine intelligence plant one could actually build?
- Can we decouple intelligence from energy consumption?
- Does future AI infra need vertical integration across power + datacenter + networking + compute?
- Which innovations move the needle enough to alter datacenter economics rather than just improve benchmarks?
Mathematics: What deep structures govern numbers?
- Can results usually taught using higher machinery be reconstructed from first principles?
- What makes a problem "easy to state but extremely hard to prove"?
- Which combinations of famous constants (like e and π) are known to be algebraically independent?
- What is the right distinction between density, probability, universality, and existence?
- How do generating functions encode arithmetic information?
- How are continuous functions and discrete mathematical objects related to each other?
- What makes Riemann Hypothesis so difficult to prove?
- How does turning an equation into a geometric object help solve Diophantine problems?
- Why are rational points on curves so difficult to understand?
- When are two mathematical objects "really the same" despite looking different?
- How much of mathematics is just about identifying the invariants that completely describe an object?
- What do paradoxes like Banach–Tarski tell us about infinity, measure, and the axiom of choice?
- How closely are topology and set theory connected?
- Why do the same mathematical objects (primes, groups, symmetries, L-functions) keep appearing in unrelated areas? Are those connections just useful analogies or evidence of some deeper structure?
- Can computation expose patterns that can suggest new theorems vs just verify examples?
- Could computational mathematics provide a more accessible frontier?
AI for science: How do we teach machines to do science?
- How should scientific knowledge be encoded so agents can reason over it?
- How do we teach machines to run the process of science vs just producing the results?
- Could continual-learning systems accumulate scientific knowledge without repeatedly retraining foundation models?
- What becomes possible when machines can search the space of undiscovered relationships across disciplines rather than merely retrieve published papers?
Here are the open source tools I've built explore the AI infra stack:
pforge: CLI to shape and serve open models on your own GPU
phabitat: CLI to give every AI agent its own computer
psplice: CLI to perform runtime surgery on foundation models
pscope: CLI to scan your hardware and tell you what models can actually run well
pworlds: CLI for verifiable RL environments
Compute: How to maximize intelligence per unit of input resource?
- What is the minimum physical work required to generate one useful token?
- What is the minimum mathematical work required to generate one useful token?
- How much of inference is fundamentally necessary versus an artifact of today's transformer implementation?
- Where does energy actually go when producing a token?
- What is the minimum achievable joules/token?
- Are FLOPs even the right measure? Or should we care much more about bytes moved / tokens per joule?
- Could we get another 100x efficiency improvement without sacrificing model capability?
Memory movement: Is moving data the real enemy?
- Why are we moving trillions of bits around just to perform relatively cheap arithmetic on them?
- Will memory movement continue to remain the dominant cost in AI systems?
- What are all the ways to avoid moving weights around?
- How far can sparsity reduce memory reads without degrading model performance? If so, on which models/ datasets/workloads?
- On the memory axis, what's going to fundamentally change the scaling curve? Is it compute-in-memory, near-memory compute, caching, quantization, sparsity, recomputation, locality, something else?
Chips: Do we need a different physical substrate?
- What would AI hardware look like if designed from first principles around energy rather than FLOPs?
- Are GPUs the right machine for intelligence or just the machine we happened to have?
- What type of physical substrates can deliver energy savings? Will it require completely new models and training procedures?
- Can existing foundation models be mapped onto radically different hardware or should models be co-designed with the chip?
- Is there a genuine "hardware lottery"? Do algorithms succeed partly because available hardware happens to favor them?
Models: How much of today's architecture is fundamental?
- What does "same intelligence with less computation" actually mean mathematically?
- What is the irreducible algorithm for intelligence vs baggage inherited from how LLMs evolved?
- Can you separate model efficiency from compute efficiency or are they inherently coupled?
- Why transformers?
- What parts of transformer computation can be removed?
- What exactly has to happen for a model to produce intelligence?
- Does an alternative architecture need to train its own foundation model from scratch?
- How much can we sparsify? What happens if only a tiny fraction of neurons fire?
Memory + Learning: Where should knowledge live?
- Is memory becoming a distinct layer alongside the foundation model and agent/coding harness?
- Is continual learning just another form of post-training or something fundamentally different?
- Should an intelligent system's knowledge be encoded in its weights at all?
- When should information live in context vs live in external memory vs consolidated into weights?
- How can a system continually update knowledge without catastrophic forgetting?
- Can external memory remain structured and attributable even when the underlying model changes?
Reasoning: What actually constitutes machine intelligence?
- Does scaling next-token prediction genuinely create reasoning?
- Why are language models so good at coding and formal symbolic tasks?
- Where does language-only learning break down?
- Do spatial, temporal, causal, and physical reasoning require fundamentally different representations?
- How do we distinguish reasoning from sophisticated pattern completion?
- What infra is required for verifiable reasoning?
- Can agents become reliable through verifiers + evaluators + execution environments + search + control loops even if the underlying model remains imperfect?
World models: Does intelligence need to simulate reality?
- What's the difference between intelligence that describes the world and intelligence that can predict/interact with it?
- Are world models a major architecture beyond language models?
- How should machines represent dynamics rather than sequences of tokens?
- Why neural operators? Are they important because they learn mappings between entire functions/physical fields rather than finite-dimensional vectors?
- Could physical-world models become foundational models for science and engineering?
- Is simulation + RL the route from passive models to systems that learn, reason, and act?
AI systems: Where does the model end?
- Will neural network continue to remain the unit of intelligence?
- Is the real product the model or the model + memory + tools + runtime + verifier + environment?
- What belongs in the weights versus the surrounding system?
- How should inference-time compute be allocated?
- Can additional search/computation at inference substitute for larger models?
- What infra will long-running autonomous agents require?
- What is the equivalent of an operating system for machine intelligence?
Power: Does AI ultimately become an energy problem?
- What do "machine intelligence plants" look like?
- What is the smallest economically viable machine intelligence plant one could actually build?
- Can we decouple intelligence from energy consumption?
- Does future AI infra need vertical integration across power + datacenter + networking + compute?
- Which innovations move the needle enough to alter datacenter economics rather than just improve benchmarks?
Mathematics: What deep structures govern numbers?
- Can results usually taught using higher machinery be reconstructed from first principles?
- What makes a problem "easy to state but extremely hard to prove"?
- Which combinations of famous constants (like e and π) are known to be algebraically independent?
- What is the right distinction between density, probability, universality, and existence?
- How do generating functions encode arithmetic information?
- How are continuous functions and discrete mathematical objects related to each other?
- What makes Riemann Hypothesis so difficult to prove?
- How does turning an equation into a geometric object help solve Diophantine problems?
- Why are rational points on curves so difficult to understand?
- When are two mathematical objects "really the same" despite looking different?
- How much of mathematics is just about identifying the invariants that completely describe an object?
- What do paradoxes like Banach–Tarski tell us about infinity, measure, and the axiom of choice?
- How closely are topology and set theory connected?
- Why do the same mathematical objects (primes, groups, symmetries, L-functions) keep appearing in unrelated areas? Are those connections just useful analogies or evidence of some deeper structure?
- Can computation expose patterns that can suggest new theorems vs just verify examples?
- Could computational mathematics provide a more accessible frontier?
AI for science: How do we teach machines to do science?
- How should scientific knowledge be encoded so agents can reason over it?
- How do we teach machines to run the process of science vs just producing the results?
- Could continual-learning systems accumulate scientific knowledge without repeatedly retraining foundation models?
- What becomes possible when machines can search the space of undiscovered relationships across disciplines rather than merely retrieve published papers?
Here are the open source tools I've built explore the AI infra stack:
pforge: CLI to shape and serve open models on your own GPU
phabitat: CLI to give every AI agent its own computer
psplice: CLI to perform runtime surgery on foundation models
pscope: CLI to scan your hardware and tell you what models can actually run well
pworlds: CLI for verifiable RL environments