Big Data Architect (Cloudera, Spark)

TL;DR

Big Data Architect (Cloudera/Spark): Designing and optimizing scalable data architectures and Global Data Lake systems with an accent on data processing pipelines and DaaS API solutions. Focus on enhancing system performance, implementing robust data governance, and leading the technical direction for international data initiatives.

Location: Madrid, Spain

Company

hirify.global is a global provider of data-driven solutions, utilizing its Sustainable Estimatics suite to optimize resource usage and reduce environmental impact in the industry.

What you will do

• Design and implement scalable data architectures using Cloudera and Spark to support global data processing pipelines.
• Lead the development of API solutions to facilitate Data as a Service (DaaS) for internal and external customers.
• Establish best practices for data ingestion, transformation, and storage across international locations.
• Optimize Spark jobs and data workflows to meet strict latency and throughput requirements.
• Collaborate with data scientists and analysts to ensure reliable datasets for machine learning models.
• Implement data governance practices and ensure compliance with international data privacy and security regulations.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
• At least 7 years of experience in big data architecture, specifically with Cloudera and Spark.
• Proficiency in Scala, Python, or Java for implementing complex data solutions.
• In-depth knowledge of Hadoop ecosystem, including HDFS, Hive, Impala, and HBase.
• Expertise in designing and optimizing data pipelines using Spark Streaming or Spark SQL.
• English: B2+ required for effective communication with international stakeholders.

Nice to have

• Experience with cloud platforms like AWS or Azure.
• Knowledge of containerization technologies such as Docker or Kubernetes.

Culture & Benefits

• Opportunity to work with the industry-leading Sustainable Estimatics suite.
• Collaborative and innovative environment within an international engineering group.
• Focus on sustainability and driving positive environmental and societal impact.
• Agile development culture with continuous integration, deployment, and architecture reviews.