Folding Data #30

Why Can't You Pull a Number Real Quick?

‍

Nobody said it was easy
No one ever said it would be this hard

Even at the most data-driven companies, data workflows leave much to be desired, and seemingly straightforward asks often lead into rabbit holes. Introspection is important not only to help our stakeholders develop empathy for our data people's lives but to improve how we work, and know what to require from our tools throughout the data value chain.

At the data creation/preparation stage, we want our tools to help us do the right thing with minimal friction. dbt simplifies building data pipelines fast while following the proven best practices of software development. Datafold takes the pain out of testing data pipelines so that you don't need to choose between hours of tedious validation vs. pushing to prod and hoping that nothing breaks.

At the data consumption stage, it gets even more interesting. It seems clear that we've exhausted the potential of the decades-old data cube concept, and to meaningfully get better at synthesizing insights from the data, we need to move to a different level of abstraction, and perhaps start thinking about the data in a different way. For instance, not as a dimensional model but as event sequences. But before we get to see and to judge the next generation BI, let's take the moment to acknowledge where we are now to know immediately when the bright future comes.

Why can't we pull that number real quick

Tool of the week: Freshpaint

I rarely get a chance to evaluate the tools first-hand in production before I write about them, but this one is actually a part of our data stack at Datafold!

Some tools in your data stack don't need to be very engaging – they just need to work well. One example is event collection and integration – I want it to be easy to set up and then just to quietly collect and send events to all the required destinations without having to worry about the scale and, hopefully, not stressing too much about the price. Another important quality of an event integration solution is the ability to make our engineers and marketers (very different as buyers and users!) equally happy with it.

For a while, the answer was Segment, but turns out that you can do better: we switched from Segment to Freshpaint ~1 year ago for the much better value/price, and because of its Autotrack feature that captures everything happening on a webpage without requiring spending valuable engineering time on instrumentation.

Let's get those events in

Before You Go