Integration Engineer (Databricks)

TL;DR

Integration Engineer (Databricks): Building and maintaining third-party data integrations for a Master Data Management platform with an accent on PySpark streaming pipelines and backend service orchestration. Focus on end-to-end development, managing external vendor relationships, and ensuring reliable data enrichment at scale.

Location: Must be based in Medellin, Colombia (Hybrid)

Company

A global staffing company providing specialized talent for a Databricks-native SaaS platform focused on MDM, relationship graphs, and PIM.

What you will do

• Design, build, and maintain end-to-end third-party data provider integrations.
• Develop PySpark Structured Streaming pipelines on Databricks for data ingestion and transformation.
• Build Python/FastAPI backend services to orchestrate API calls and manage rate limiting.
• Create Node.js/React frontend interfaces for integration configuration and monitoring.
• Act as the primary technical point of contact for third-party data vendors.
• Ensure system reliability through robust error handling, logging, and alerting.

Requirements

• 5+ years of experience in application development and data engineering.
• Strong Python skills with production experience using FastAPI.
• Practical PySpark experience including Structured Streaming and Delta Lake.
• Node.js and React proficiency for frontend configuration tools.
• Proven track record integrating with third-party REST APIs and handling complex authentication.
• Excellent English communication skills for direct vendor collaboration.
• Solid SQL skills and experience with relational data modeling.

Nice to have

• Experience with enterprise data enrichment providers like ZoomInfo or Dun & Bradstreet.
• Familiarity with Databricks Unity Catalog and Workflows.
• Experience with Azure or AWS cloud environments.
• Exposure to CI/CD, Docker, and Terraform.

Culture & Benefits

• Global staffing company with international reach.
• Opportunity to work on a cutting-edge Databricks-native platform.
• Focus on ownership and end-to-end feature delivery.
• Professional environment valuing consistency and technical reliability.