Senior Data Engineer

Data Engineer

We are seeking a talented and experienced Data Engineer with expertise in Azure Databricks to join our innovative team. In this role, you will play a pivotal role in building and optimizing data processes using cutting-edge technologies like PySpark/Scala or Spark SQL within Databricks notebooks. You will craft and fine-tune data factory pipelines to seamlessly ingest, process, and load data from diverse sources into our data lake and Synapse environments. Additionally, you will utilize your expertise to design and implement database structures, including tables, views, and stored procedures in Synapse.

Responsibilities:

• Build and streamline data processes using cutting-edge technologies like PySpark/Scala or Spark SQL within Databricks notebooks.
• Craft and fine-tune data factory pipelines to seamlessly ingest, process, and load data from diverse sources into our data lake and Synapse environments.
• Utilize your expertise to design and implement database structures, including tables, views, and stored procedures in Synapse.

Required Skills:

• Minimum of 3 years of hands-on experience constructing pipelines with Azure Data Factory, ensuring data flows smoothly and efficiently.
• Proficiency with Spark on Databricks, with over 2 years of experience leveraging PySpark/Scala or Spark SQL to unlock the full potential of your data.
• Comfortable coding in languages like C#, Java, or Python, adding versatility to your toolbox.
• Over 5 years of experience navigating databases and ETL tools, honing your skills to perfection.
• Primary Skill: Azure Databricks - Your expertise here will be the cornerstone of our data engineering endeavors!