Back to all stories

Testing Spark applications to improve quality

  • Adevinta Life
  • Tech

Developers at Adevinta Spain wanted to run unit, integration and end-to-end tests on Spark applications. Learn how they implemented this.

A useful read for big data developers, get into the detail of how the team at Adevinta Spain implements unit, integration and end-to-end tests on Spark applications as they build a Data Platform. The full article reveals a typical project layout, how to use Spark sessions, what the various test functions enable and how this improves the quality of the code and the CI pipelines.

Highlights include:

  • Layout of a Spark project
  • How to isolate different Spark tests
  • Details of the test implementation code
  • The idea behind SharedSparkSessionHelper
  • Links to real-life Adevinta Spain examples

Related stories

Discover all stories

Check out what our Brand Ambassadors in Spain and Benelux achieved in Q1 2024

Read more about Check out what our Brand Ambassadors in Spain and Benelux achieved in Q1 2024
what our brand ambassaors achieved in Q1 in Spain

International Day of Persons with Disabilities – Initiatives across Adevinta

Read more about International Day of Persons with Disabilities – Initiatives across Adevinta

How we celebrated Black History Month in Adevinta

Read more about How we celebrated Black History Month in Adevinta