The Evolution of Modern Data Stack by Derek Steer

In our first-ever user conference, Derek talked about the history of the modern data stack, and where it could go in the future. He began with the idea that many SaaS businesses exist because of the modern data stack due to the influx of innovation.

One example Derek shared is how video game companies like Zynga adopted a strategy of combining their product activity data and purchasing data to design products for maximum revenue generation — a framework that modern B2B companies have since then copied.

Derek then highlights data innovations in the past 15 years: Big Data, The Cloud and the rise of data science.

​All of these pivots have paved the way for a big shift — the commoditization of data storage and processing. Open source tools like Hadoop weren’t supported by vendors, and administering these tools became too big of a burden aside from big companies that could afford the costs. Today, startups or smaller companies could perform the same tasks in a more accessible way. To win the modern data stack, he suggests the following:​

1. ​Design your data stack to be as modular as possible.

2. Ensure that you are tracking events for any key action, and that events are easily usable in any system that needs them.

3. Optimize the system, not each component.

4. Design with the newest advancements in mind.