When it’s time to set up a data stack, there are three paths you can take. You can build it, buy it, or outsource it to a contractor or agency. If you’ve never done this before, choosing the right option for your business can be difficult.
Jessica Larson, Data Engineer at Pinterest, and Mike Ouye, SVP Product and General Manager at Cloud9 Esports, shared their experiences and advice on how they’ve made this decision in the past.
Five factors that will inform your decision
There are five things that will either make the decision for you or steer you in the right direction:
Your company’s core competency
Ouye prioritizes looking at your company’s core competency to inform whether to buy or build. He strongly believes it doesn’t make sense to invest in building tools that don’t align with your core competency.
Ouye has worked at numerous gaming companies that were extremely data-driven, and a lot of their decisions were made based on the core competency, speed, and size of the company. “For instance, at Cloud9, we’re an esports team. We’re mainly driven by our teams — League of Legends, Valorant, etc. — and we operate very much like a pro sports team. So at our core competency, analytics is not one of them. We’re also not a very engineering heavy team, so I think those two factors really made the decision [to buy] for us,” said Ouye.
If your company’s core competency isn’t data or analytics, buying your data stack makes the most sense. This is because you’ll divert precious resources from your core competency, and the quality of the data stack you build won’t be comparable to what’s already on the market.
Type of data that’s being handled
At Pinterest, Larson works with product, sales, HR, and finance data. It’s extremely sensitive data, so the number one thing they think about is security. They need to make sure data isn’t exposed to additional eyes at any point, starting with extracting data from the source to the data warehouse and then all the way through to any downstream tools.
When you’re handling confidential data and security is very important, building a data stack ensures third-parties don’t have access to your data. You have complete control over your data and can implement the privacy and security measures you want.
Even if you’re handling sensitive data, don’t rule out buying a data stack. Although Cloud9 deals with highly confidential consumer data, business analytics, and competitive data and security across their entire stack is very important, they don’t shy away from working with vendors. Instead, they look for people who have experience working with the types of data they process and who they have a high level of trust in.
Your tech stack
If your tech stack is mostly made up of out-of-the-box solutions and you’re using them in a straightforward way, you’ll be able to buy your data stack and integrate it into your tech stack. But if you’re using custom-built systems or using technology in a unique way, you’re usually better off building. This is because it’s hard to buy a solution that will work with all your tools or support the unique way that you want to use your data stack.
Organizational preference
Between building or buying, Larson says your choice will depend on whether your organization typically leans toward one or the other. What is your organization set up for? If your company typically builds its own tools, you’ll probably have a large engineering team to support internal tooling.
Budget
There’s never a buy, build, or outsource decision that’s made without considering budget. The most expensive option is hiring internally or using employees to build your data stack. The cheapest option is to buy an existing solution. Hiring a contractor falls somewhere in the middle.
In thinking about budget, Ouye refers back to core competency. When he was working in free-to-play games in the past, data was a huge part of the business and they had large data sets, so it made sense to hire an engineering team to build their data stack.
At Cloud9 though, “For us, we’re a competitive esports organization. We’re not an analytics company, nor are we a data company. So for our case, it just doesn’t make a lot of sense to go out and hire a bunch of developers and engineers,” said Ouye. “We’re really looking for people and companies who do this a hundred percent of the time, like Mozart for instance. All you guys do is analytics. I think that’s great for us because we’re getting best-in-breed, best-in-class.”
While outsourcing the work may seem like the best balance between cost and quality, Larson thinks it’s tricky to do so. “Thinking about contracting, that does tend to be a cheaper way to go, but that’s also its whole own can of worms. How do you find the right project that can be given to a group of contractors that could leave at any time — more so than an employee might. And you might have a little bit less control over that whole process when you do that.”
If you’re at a company where there are separate budgets allocated for tooling and headcount, this can make your decision easier. Pinterest has more budget for tooling than staffing. This makes it a pretty easy decision to choose to buy a solution. “I think a lot of these tools are going to in the long run be cheaper than headcount, especially as we see engineers are not cheap these days,” added Larson.
Can you put off implementing a data stack instead?
Aside from buying, building, and outsourcing, there’s actually a fourth option — punting the decision. When does it become mission-critical to have a data stack and when is it okay to punt it?
Very early-stage start-ups can delay setting up a data stack. “When you’re pre-product or pre-launch, you don’t need a big data stack. You should really be focused on your product. And similarly, when you’re after launch or doing a beta or alpha, most of the really key metrics that you need can be done in Excel or Google Docs,” said Ouye.
For Larson, a data stack becomes a must-have when you have problems that need to be solved with data and when the volume of data is so large that you can’t just look at it and find trends or insights yourself. A data stack also becomes crucial when you have a data set that’s large enough for you to do statistical analysis on.
If you’re still looking for more guidance on whether to buy, build, outsource, or punt your data stack, hear directly from Larson and Ouye in our full discussion with them.