What Data Practitioners Wish They Knew Before Their Migrations

Data migrations rarely fail because of a single catastrophic error. More often, they unravel due to hidden complexity, overlooked dependencies, and misaligned expectations—problems that only surface when it’s too late to course-correct.

We spoke with seven data practitioners who have led large-scale migrations across industries like healthcare, fintech, insurance, and nonprofit. Their experiences reveal a hard truth: migrations are more about people than data.

The biggest challenges weren’t about moving records from one system to another. They were about:

• Untangling undocumented logic buried in legacy systems
• Avoiding common traps that lead to stakeholder frustration
• Automating validation before small discrepancies snowball into major failures

Here’s what they wish they had known before starting their migrations—in their own words.

"If your migration takes too long, you risk the platform you’re moving to becoming legacy before you even finish. It’s a double disaster." — Gleb Mezhanskiy, former Data Engineer at Lyft & CEO of Datafold

Gleb led a multi-year migration at Lyft that spiraled out of control when scope expanded beyond just moving data. Instead of a straightforward lift-and-shift, his team tried to redesign the data model and optimize everything mid-migration—a decision that ultimately delayed progress and created more risk.

By the time they finished migrating to Hive in 2020, the entire industry had already moved on—leaving them stuck on a system that was already outdated. 

Read more to learn why Gleb argues that speed isn’t just about efficiency—it’s about avoiding obsolescence. Move quickly so you don’t find yourself stuck maintaining a system that no one else wants to use.

"Figure out how best to scale with the knowledge that things will still come up that you need to fix, or issues will still need to be looked into. There is no perfect time to scale." — Ryan Hawkins, Senior Analytics Engineer at the Children's Hospital of Philadelphia

Ryan’s team migrated a 15-year-old on-prem Netezza system to Snowflake, but one of their biggest lessons wasn’t about infrastructure—it was about timing. Initially, they kept the migration team small, assuming they could bring in more engineers later once they had ironed out the process. But by the time they scaled up, it was already too late to catch certain issues early.

Because they waited too long to involve more people, critical problems only surfaced when it became much harder to fix them.

Read more to learn why Ryan believes early scaling is key to migration success—how his team navigated tangled dependencies, automated incremental validation with GitHub Actions, and built an in-house tool to ensure parity across thousands of tables.

"Sometimes there were even conflicting requirements…listed out on a document, but when you look through the stored procedures, it didn’t exactly seem to align." — Jasmin Tan, Staff Data Analyst at Brooklyn Data Company

Jasmin joined a nonprofit’s migration project with a clear mission: move critical campaign logic from a third-party vendor to an in-house Redshift and dbt setup—without breaking business continuity. But translating legacy logic wasn’t as simple as it seemed. 

The project required reverse-engineering undocumented business logic, often piecing together hidden transformations and unspoken assumptions from stored procedures. Without a clear source of truth, validation became an intensive process—so much so that the nonprofit developed a custom R script to compare outputs.

Read more to learn how Jasmin tackled undocumented logic, validated business-critical transformations, and navigated the complexities of nonprofit data migration.

"It’s easy to run a quick test and confirm that a tool works, but when you start to factor in the scale of what you’re dealing with, things can be slightly different, so you have to spend a lot of time understanding the limitations." — Emmanuel Ogunwede, Data Engineer at Brooklyn Data Company

Emmanuel’s team tackled a zero-downtime ETL-based migration for a live transactional database, handling complex entity resolution, schema drift, and performance constraints in an event-driven architecture. But one of his biggest lessons? A migration that works in small tests can still break at scale.

One of the toughest challenges was schema drift detection—hundreds of data sources, each structured differently, required continuous validation to prevent silent mismatches.

Read more to learn why Emmanuel emphasizes scaling considerations in migrations, how his team solved entity resolution challenges without external tools, and the engineering behind a zero-downtime ETL migration.

"It’s hard to see why the process doesn’t need to be so complex from the inside, whereas we’re seeing it from a completely fresh perspective, we can immediately say, ‘this doesn’t need to be like this at all.’" — Fabio Biondolillo, Senior Analytics Engineer at Brooklyn Data Company

Fabio’s first major migration project wasn’t just about moving data—it was about challenging deeply entrenched processes that had been patched together over years. As an external consultant, he quickly realized that the biggest obstacles weren’t technical but cultural: teams were hesitant to change workflows that had “always worked,” even if they were inefficient.

Without much existing documentation or institutional knowledge, the team had to reverse-engineer workflows, optimize Power BI dashboards, and refactor SQL-heavy stored procedures into modular dbt models—all while ensuring that downstream stakeholders didn’t experience disruptions.

Read more to learn how Fabio approached untangling undocumented workflows, phased a migration for minimal disruption, and leveraged consulting to bring fresh perspective to legacy system refactoring.

"People don’t want to put any resources into actually monitoring, measuring and handling data quality.v — Alex Meadows, Data Architect at Sorcero

With five data migrations under his belt, Alex has seen the same failure pattern repeat across industries: companies treat migration as a technical challenge, but ignore the critical role of data quality.

While teams invest in cloud infrastructure, pipeline refactoring, and DevOps, data validation and quality controls remain an afterthought—if they’re considered at all. The result? Broken business processes, lost trust in analytics, and migrations that appear successful on the surface but fail in practice.

In Alex’s experience, the most successful migrations embed quality checks at every level—from pipeline validation to true parity testing between legacy and new systems.

Read more to learn why Alex believes data quality should be a first-class priority, how lift-and-shift strategies often fail without it, and why organizational culture matters more than tooling in migrations.

"Validation, as an umbrella across the entire process, was definitely the biggest challenge." — Sandro Palleschi, Director of Enterprise Data Services at Empire Life

Sandro led a migration away from Informatica, which involved a full data model redesign—creating both technical challenges and opportunities to improve data quality. But the biggest hurdle wasn’t just schema changes or performance tuning—it was validation.

With a completely different schema structure, traditional data parity checks weren’t feasible. Instead, his team had to develop custom validation strategies, ensuring every row in the new system produced the same results as the legacy platform, despite fundamental transformations.

‍Read more to learn how Sandro tackled large-scale data validation, handled schema mismatches, and ensured regulatory compliance in a high-stakes migration.

It's all in the details: Why migrations succeed (or fail)

Every expert we interviewed reinforced the same truth: migrations fail when they’re treated as purely technical exercises.

The hardest problems aren’t always in the SQL or the pipelines—they’re often in misaligned expectations, poor validation, and overlooked dependencies.

Learn from those who’ve been through it:

• Anticipate the mess
• Automate (and actually do!) validation
• Document rigorously
• Communicate proactively

That’s what separates successful migrations from painful, months-to-years-long firefights.