Senior Databricks Engineer

Key Responsibilities

• Design, develop, and maintain scalable and robust data pipelines that collect, process, and store large volumes of structured and unstructured data.

• Architect and implement data warehouses, data lakes, and other storage solutions that support analytics and reporting needs.

• Optimize data architectures and workflows for performance, scalability, and cost-efficiency.

• Collaborate with data scientists and analysts to understand data requirements and ensure that data systems meet their needs.

• Ensure data quality, integrity, and security by implementing best practices in data governance and management.

• Develop and maintain documentation for data systems, including data models, flow diagrams, and operational procedures.

• Mentor and guide junior data engineers, providing technical leadership and support.

• Stay current with emerging technologies and trends in data engineering, and apply them to improve existing systems.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.

• 5+ years of experience in data engineering or related fields, with a focus on large-scale data systems

• 3+ years of experience with Databricks

• Proficiency in SQL and experience with databases such as MySQL, PostgreSQL, or NoSQL databases

• Strong programming skills in languages such as Python, Java, or Scala.

• Experience with ETL tools and frameworks and data pipeline orchestration.

• Familiarity with big data technologies (e.g., Hadoop, Spark, Kafka) and cloud platforms (AWS, GCP, Azure).

• Strong understanding of data warehousing concepts, data modeling, and schema design.

• Excellent problem-solving skills and attention to detail.

• Strong communication skills and ability to work effectively in a team-oriented environment