How does data become intelligence?

It all starts
with the data.

Superintelligence will take on the most consequential scientific questions of our time, transforming the pace of progress and the limits of human capability. Much of that progress will come from working alongside increasingly capable models. But a model is a reflection of the data it is trained on. While a lot of research examines architectures, training methods, and model optimizers, everything that a model could theoretically learn is governed by its data and experiences.

If we can understand how data gives rise to model capabilities, we can dramatically expand our ability to shape, control, and customize intelligence. In the last few years, data curation methods such as filtering, mixing, and synthetic generation have become major levers of progress, but the community’s understanding of the science of data has not improved at the same rate.

The science of data sits orphaned between the people resourced to study it and the people motivated to share it. The organizations with the data, compute, and infrastructure to study data at scale have no incentive to disseminate their research. Academic researchers have the incentive to publish but rarely the resources to run data experiments at breadth and scale. Few places have both.

Data research is the soul of DatologyAI. Pursuing our mission has allowed us to consistently innovate and develop greater understanding of the science of data, while also teaching us how much still remains unexplored. Today, we're expanding that commitment with Datology Labs, our open research initiative dedicated to defining the science of data. We believe ambitious science is best pursued collaboratively and in the open. Datology Labs will bring together talented researchers from Datology, academia, and beyond who share an obsession with data, and give them the freedom to pursue some of the most meaningful work of their lives.

Defining the science of data.

The more we learn about data, the more questions we discover. Some of these questions stem from the fundamental relationship between data and learning, while others challenge how we think about safety, alignment, and entirely new scientific domains. We are sharing a non-exhaustive list of data problems that we’re excited about.

Studying the data life cycle.

Data has a life cycle that extends beyond preparing a training dataset. It is sourced and processed, then ordered and combined across stages of training. As a model learns from that data, information is absorbed into its weights, where it interacts with what the model has learned before and with the data it encounters later. These interactions ultimately shape the capabilities and behaviors that emerge. Models also inherit information from other models through distillation and synthetic data. Their capabilities and behaviors shape what data they produce and what humans create from them. Models write data back into the world, which can become part of the next model's training data.

We want to understand these interactions across the entire life cycle. Why do some examples reinforce previous learning while others interfere with it or cause forgetting? How does data order affect learning and training stability? How does data seen in different training stages interact with each other? And how should we curate data for distillation and synthetic generation to preserve and build on what previous models have learned?

Understanding how data shapes learning.

One of our bets is that it is possible to predict what a model will learn before we spend the compute to train it, through the in-depth study of data. Somewhere in a model's weights, data transforms into a model’s knowledge, capability, and behavior, but we still don't fully understand how. Understanding where information is represented and how behaviors and capabilities arise from training examples could help us close the gap between what data contains and what models are able to learn from it, allowing us to train models that are significantly more data efficient than the current generation.

To get there, we need new ways to study data itself. What makes some data more useful than others? Can small-scale experiments predict large-scale model behavior? We want to develop measurements and visualizations that help answer these questions, alongside attribution methods that connect capabilities to training examples and forensic methods that investigate data provenance through model weights.

Safety and alignment through data.

Building superintelligence that is safe, faithful, and honest is one of the hardest problems of our time. Both safe and unsafe behaviors are learned, and what a model learns comes from its data and experiences. Understanding how data shapes these behaviors will be a central goal of our research.

Where do safe and unsafe behaviors live inside a model? What data curation properties can help make safety inseparable from a model's useful capabilities? Does the timing of a data intervention change its robustness? Can seemingly benign data contain natural poisons that compromise safety? Can we identify the examples responsible for an unsafe behavior and design interventions that prevent it?

Taking the science of data to new domains.

Machine learning research has followed the same pattern again and again. When entering a new domain, the field rushes to models, architectures, optimizers, and training techniques. This has played out across vision, language, time-series, and multimodal systems. Enormous effort often goes into improving how models learn before we develop an equally rigorous understanding of what they learn from.

The next phase of AI systems is reaching fields such as physics, materials science, and biology. Many of these fields are starved for data, which makes data curation matter more, not less. Everything we have learned about data, from classic algorithms to synthetic data generation, is waiting to be tested there. Which principles of data curation carry over to these fields, and which need to be rethought from scratch?

Work with us.

We are excited about the unknown and the uncertain. We want to follow the unanswered questions and unexpected discoveries that come from data.

Spend time on a frontier data research problem as a Datology Labs fellow. All we ask is that you are obsessed with data and willing to take risks.