How Healthy Directions pulled off a successful data migration to Snowflake with Datafold

Data migrations often fill data teams with dread due to the endless complications they can bring: delays, data discrepancies, and operational disruptions. When Healthy Directions set out to migrate their data warehouse from SQL Server to Snowflake, they found that Datafold provided unmatched quality assurance. By adopting Datafold’s data diffing capabilities, the data engineering team ensured high-quality data controls throughout the most complex and critical stages of the migration.

Healthy Directions

Healthy Directions is a direct-to-consumer retailer of health supplements and skincare products. They collaborate with integrative and alternative health professionals to create doctor-formulated nutritional supplements aimed at supporting various health and wellness needs. Founded in 1974, Healthy Directions advocates for a holistic approach to wellness and is dedicated to providing high-quality health products and exceptional customer service.

The challenge: Migrating to the cloud while ensuring business continuity

Healthy Directions embarked on a critical project to transition their data warehouse from an on-premises SQL Server to Snowflake. This was part of a broader strategy to optimize their IT infrastructure and reduce operational costs while enhancing their data processing and analytics capabilities.

The migration was complex as the data team needed to accurately replicate stored procedures and SSIS packages within Snowflake’s architecture. This required the data engineers to not only translate existing data transformation processes, but also optimize them to leverage Snowflake’s advanced features effectively.

The team needed to ensure that both legacy and new systems were at parity, and that the new processes performed optimally to meet existing operational demands.

A major challenge was validating data integrity on both sides of the migration. This was an especially difficult quality pain point, and highlighted the need for a solution that enabled rigorous data validation on both sides.

The solution: Datafold’s cross-database data diffs are integral to successful migrations

Datafold played a critical role in ensuring that hundreds of objects were successfully and accurately copied from SQL Server to Snowflake. Cross-database data diffs empowered Healthy Directions’ data team to streamline the validation process, reducing manual effort and room for error.

Validating every object with Data Diff

Validating between source and target databases was crucial, as discrepancies needed to be identified and resolved quickly to maintain the reliability of business operations. Using Datafold’s precise data diffs, Healthy Directions’s data engineers could pinpoint specific differences between the SQL Server and Snowflake data at the value-level, ensuring that any differences could be caught and fixed. The team quickly identified and resolved discrepancies during the migration without needing to write extensive SQL code to compare objects. They estimate that Datafold saved them 4-5 weeks of manual validation time. This was crucial for validating that the migrated data was accurate and complete, helping to prevent potential issues in business operations due to data inconsistencies. 

A more efficient migration process

By taking the guesswork out of pinpointing where discrepancies occurred, Healthy Directions’s data team could streamline and automate their data validation efforts. This precision allowed the team to focus directly on any anomalies that surfaced, and shortened a typically lengthy, manual, and tedious data validation process. As a result, the migration was completed right on schedule, even with an aggressive timeline. Datafold was thus integral to significantly reducing the time and effort required to ensure data consistency and accuracy during the migration. 

The results: An error and delay-free data migration

A smooth and stress-free migration

Most migrations run well past budgets and project deadlines because it’s hard to translate between SQL dialects and validate that each translated job still produces the same data. Datafold’s automation of complex validation tasks meant that Healthy Directions’s data engineers did not need to create and perform manual checks that still left room for error. The data team could focus their efforts on making sure that other critical aspects of the migration proceeded smoothly. During the migration, Healthy Directions experienced 0 unplanned operational downtime.

Stakeholder trust from day one

By integrating Datafold into their migration strategy, Healthy Directions could continuously monitor data integrity. This ongoing validation was essential for maintaining operational continuity, as it ensured that the business could rely on the new system from day one, without disruptions or data-related errors.

Beyond the immediate migration, Datafold's capabilities in data diffs, CI/CD testing, and monitoring positioned Healthy Directions to maintain high data quality standards in the future. As they continue to scale or potentially integrate new data sources, Datafold's tools can ensure that data integrity remains a constant in their data engineering workflow, supporting more data-driven decisions without compromising on reliability.