The future of data engineering is AI-powered

Data engineers face a striking paradox: while they're tasked with building increasingly sophisticated data products, most of their time is spent on manual, repetitive tasks rather than meaningful innovation. Broadly, developers spend surprisingly little time writing code—the rest is consumed by tedious work like code reviews, testing, and documentation.

For data engineers, the challenge is even more acute. The modern data stack's complexity means engineers spend countless hours investigating data quality issues, debugging pipeline failures, and validating code changes. Teams undertaking data platform migrations can spend months or even years locked in cycles of code translation and validation.

From the beginning, Datafold has been focused on eliminating manual toil. We gave data engineers automated CI testing they could trust by showing them exactly how their code changes would impact their data and downstream assets in production. Then, we automated 90% of the manual work of data platform migrations. Now, we're bringing AI-powered automation to one of a data engineer’s most critical and time-consuming tasks: code review.

Introducing AI-powered analysis for code review

Today we're announcing the private beta of three new AI capabilities that will help data teams dramatically reduce manual work while maintaining data quality:

Get to the point with automated PR summaries
‍Now every pull request tells a story: what changed, why it changed, and how those changes impact your data. Datafold's AI-based overview automatically analyzes PRs to surface the most important changes—and critically—their downstream effects, helping reviewers focus their attention where it matters most.

Get root cause analysis (RCA) for data diffs
‍When data changes unexpectedly, finding the root cause typically requires hours of investigation. Datafold's new root cause analysis capability automatically connects changes in your data back to the specific code changes that caused them. Simply hover over a change in your data diff to see exactly which lines of code were responsible.

Investigate a data change with context-aware chat
‍Sometimes you need to dig deeper to understand complex changes to your data. Our new chat interface lets you ask natural language questions about your PR or data diffs. Drawing on our comprehensive knowledge of your codebase, lineage, and data changes, conversational AI provides detailed answers that help you understand changes faster.

What sets these features apart isn't just their AI capabilities—it's how they build on our core strength of data validation. Every AI-powered insight is grounded in data diffing, ensuring that automated processes maintain data integrity. We're not adding AI for AI's sake—we're using it to meaningfully accelerate data engineering workflows while proving that data outputs remain as expected.

Bringing automation to this critical workflow delivers speed and efficiency. But combining automation with context is what drives the confidence necessary to move quickly without adding risk for the business.

The future of data engineering

The private beta of these features represents another step toward our vision of tangibly using AI to help data teams ship high quality data pipelines faster. We know that by systematically automating manual tasks, we can free data engineers to focus on what matters most: innovation, strategic thinking, and delivering business value through data.

We're excited to work closely with our beta customers to refine these capabilities and continue pushing the boundaries of what's possible in data engineering automation.

If you're interested in joining the private beta or learning more about our AI-powered features, get in touch with your rep or register your interest.

‍

‍

‍