Skip to content
Search icon

SoundCloud

SoundCloud: Amplifying Data Processing Efficiency

SoundCloud partners with Zencore for migration to Google Cloud Platform.

SOUNDCLOUD CUSTOMER SPOTLIGHT
 

Migration to Google Cloud Platform


Project Location:
Berlin, Germany
Industry:
Audio Streaming
Use Case:
Migration of Apache Spark workloads from on-premise

In the pursuit of excellence, SoundCloud recognized the need to optimize their data processing infrastructure to meet the demands of a growing user base. They embarked on a journey to migrate their Apache Spark workloads processing data on HDFS from their on-premise data center to Google Cloud.

About SoundCloud

SoundCloud, an artist-first platform empowering artists and fans to connect and share through music, is dedicated to delivering captivating experiences to listeners globally. As a trailblazer in the music industry, SoundCloud constantly pushes the boundaries of innovation, seeking cutting-edge solutions to elevate their services and provide unparalleled value to their diverse community of artists, podcasters, and music enthusiasts.

Project Challenges

SoundCloud sought to leverage scalable resources and increase cost-effectiveness by migrating to the cloud. They needed a partner with expertise in data solutions and migrations to execute this ambitious migration seamlessly.

Scope of Work

Zencore partnered with SoundCloud to migrate 28 products/workloads, comprising 200 Spark Jobs, 400 DAGs, and 100TB of data from their on-premise data center to Google Cloud Platform. The migration involved transitioning business-critical HDFS/Spark jobs to Cloud Composer and Dataproc Serverless.

Key Achievements

  • Reduced operational expenses and significantly decreased processing time
  • Aligned with SoundCloud's overarching strategy of becoming fully cloud-native
  • Enhanced scalability and flexibility of data processing infrastructure
200
Spark Jobs
400
DAGs
100
TB of Data

Solution

Zencore engineers worked hand-in-hand with SoundCloud engineers to plan and execute the detailed migration process. The goal was to adopt a "lift & shift" strategy to seamlessly transition their Apache Spark data products and Apache Airflow DAGs to GCP's Cloud Dataproc and Cloud Composer, respectively.

Technologies

  • Apache Spark
  • Apache Airflow
  • Google Cloud Platform
  • Cloud Dataproc
  • Cloud Composer

Business Goal

SoundCloud's primary goal was to optimize their data processing infrastructure, leveraging the scalability and cost-effectiveness of cloud computing. By migrating to Google Cloud Platform, they aimed to enhance their operational capabilities, reduce expenses, and align with their strategy of becoming fully cloud-native.

Business Value

The migration to Google Cloud Platform enhances SoundCloud's operational capabilities and reinforces their commitment to delivering a seamless and immersive audio experience to users worldwide. The partnership with Zencore lays the foundation for future growth and success, positioning SoundCloud to embrace the future of cloud computing and continue redefining the audio streaming landscape.

“I was extremely pleased with the performance and expertise demonstrated by Zencore throughout our migration to Google Cloud Platform. Their meticulous approach ensured a seamless transition. Zencore's partnership was instrumental in our success, and I look forward to continuing our collaboration on future endeavors.”

Jake Maizel, VP Engineering, SoundCloud
Jake Maizel | VP Engineering | SoundCloud
TELL US WHAT YOU're SOLVING FOR

Schedule a Briefing