Experimentation is a crucial part of any tech company’s growth strategy. At Adevinta in 2021, with 22 marketplaces spread across the globe (leboncoin in France, mobile.de in Germany, Marktplaats in the Netherlands and many more), experimentation was happening on many fronts—but with little cohesion. Every team had its own tool for A/B testing, resulting in inefficiencies, redundancy and escalating costs.
When I joined Adevinta, one of my first major projects was to streamline this experimentation process.
My mission: take ten different A/B testing tools used across our local marketplaces and replace them with a unified solution.
The question, of course, was whether to build our own platform or buy an existing one. This decision was not just about choosing software—it was about shaping the future of experimentation at Adevinta.
Here’s how we made the decision.
Step One: Secure Leadership Buy-In
Any major company-wide initiative needs the support of senior leadership. This is especially true when considering a build vs. buy decision, where both options can have significant implications for budgets and timelines.
We started by aligning with leadership on the goals: Did we want to save money, improve efficiency or scale experimentation across markets? After discussions with various stakeholders, we determined that the goal was to standardise experimentation across all marketplaces while also scaling our ability to test at a higher velocity.
Leadership buy-in was crucial because this decision would require resources from across the company, and ultimately, they would need to approve our strategy.
Pro Tip: When securing leadership buy-in, draft a simple one-pager outlining the project’s objectives, constraints and expected outcomes. This helps clarify the project from the start and ensures everyone is on the same page. Additionally, to support the decision-making process, include Benchmarking on experimentation tooling.
Step Two: Build a Cross-Functional Team
Experimentation touches many areas of the business—engineering, product, data, marketing and UX all have different needs. To make the right decision, it was essential to involve experts from each of these domains.
We formed a cross-functional working group with representatives from engineering, product management, data analytics and UX. This team played a key role in understanding the requirements of each department and evaluating potential solutions from multiple perspectives.
The collaborative approach not only brought in expertise from across the company, but it also ensured that everyone was invested in the outcome.
Pro Tip: Make sure to define clear roles and responsibilities in the working group. Set expectations early on about the commitment required and establish a clear decision-making process.
Step Three: Identify and Prioritise Needs
Once the team was in place, the next step was to gather and prioritise requirements for the ideal experimentation platform. The goal wasn’t just to replicate what we were already doing, but to find a solution that would improve our ability to experiment.
We identified two types of needs:
- Core capabilities: These were non-negotiables—features the tool absolutely had to have to maintain our current testing capabilities.
- Growth capabilities: These were features that would enable us to scale and enhance our experimentation efforts.
For example, core capabilities included easy test setup, automated statistical significance and reliable data integration. On the other hand, growth capabilities included machine-learning-driven personalisation and the ability to run complex multi-armed bandit tests.
Pro Tip: Keep the wish list grounded in reality. Ensure every ‘nice-to-have’ feature is tied to a concrete business use case—this helps avoid over-specification and keeps the project focused on real needs.
Step Four: Evaluate Build vs. Buy Options
With our requirements in hand, it was time to evaluate whether we should build our own platform or buy an existing solution. Here’s how we approached this:
- Buying a third-party tool would allow us to move quickly, but we’d have to evaluate vendors carefully to ensure their solutions would meet both our current and future needs.
- Building our own tool would give us more control and flexibility, but it would require significant time and resources upfront.
To make an informed decision, we created a scoring matrix to compare various options, weighing factors like functionality, scalability, integration costs, licensing fees and time to market. Each tool was scored based on how well it met our needs, with a particular focus on the must-have features.
We also considered long-term implications. A third-party tool might be a quick fix, but would it support our needs in 3-5 years? Could we evolve the platform as Adevinta grew, or would we be locked into a rigid system?
Pro Tip: When building your business case, consider how costs and needs will evolve over time. Don’t just look at your current usage—consider how scaling up your experimentation programme might change costs, and account for future growth in your projections.
Step Five: Presenting the Recommendation
After several weeks of research, collaboration and debates within the team, we developed a recommendation. Instead of choosing between building our own platform or buying a third-party solution, we proposed a hybrid approach: continue building our own platform while using a third-party tool to support the transition.
This allowed us to retain full control over the features and scalability of our own platform, while benefiting from the immediate functionality and support of a third-party tool. The third-party tool would handle our short-term needs, particularly in specific marketplaces, while we gradually developed our in-house solution to meet the company’s long-term objectives.
This approach gave us the best of both worlds:
- Flexibility: We could build a tool tailored to our unique needs in strategic areas without affecting business continuity
- Risk management: By leveraging both solutions, we minimised the risk of relying solely on one option and kept our options open.
When presenting the recommendation to leadership, we emphasised how this approach would allow us to scale our experimentation programme faster in the short term while ensuring that our internal solution was being developed.
We also highlighted the importance of long-term cost savings and strategic control that the internal platform would offer once fully built.
Pro Tip: When recommending a hybrid solution, it’s essential to clearly define the roles of each tool and outline the transition plan. Set realistic timelines for when the in-house tool will take over from the third-party solution and prepare for potential delays.
Step Six: Implementation and Beyond
Once we had the leadership’s approval, the next step was implementation. We created a detailed roadmap, including timelines for teams onboarding, integration with existing systems and training for teams.
During the rollout, we kept communication open with all stakeholders to ensure a smooth transition. Regular check-ins with leadership and team leads helped keep everything on track.
In the end, the streamlined experimentation platform allowed us to test faster, more effectively and at scale—helping Adevinta’s marketplaces make data-driven decisions more easily than ever.
Pro Tip: Continue to iterate on your solution. Even after implementation, revisit your decision regularly to validate assumptions and check for new opportunities in the tooling landscape.
Conclusion
The journey from ten tools to one wasn’t easy, but by following a structured process, we were able to make a decision that worked for Adevinta’s unique needs.
The build vs. buy debate is a challenge many companies face, but by securing leadership buy-in, building a cross-functional team, identifying and prioritising needs, evaluating options carefully and planning a strong implementation, you can navigate this challenge successfully.
Remember, there’s no universal answer to the build vs. buy question—it all comes down to your company’s specific context. However, with the right approach, you can make the right decision and set your organisation up for success in experimentation.
