SENIOR DATA ENGINEER - Databricks

Happiest Minds Technologies

Not disclosedBengaluru (On-site)
Posted: 4d agoOpening: 1

Skills Required

Key skills are marked withstar
  • Key skillData Engineering
  • Key skillSQL
  • Key skillPython
  • Apache Spark
  • Data Pipelines
  • Airflow
  • Kafka
  • Data Modeling
  • Data Warehousing
  • Snowflake
  • Databricks
  • Cloud Platforms
  • dbt
  • Data Quality
  • CI/CD

Job Description

Data Engineer Responsibilities: Design, develop, and maintain scalable data pipelines using Databricks and Azure Data Factory. Implement data processing solutions using Spark and Delta Tables to optimize performance and reliability. Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets. Manage and optimize data storage solutions in Azure Data Lake and ensure data accessibility through Unity Catalog. Monitor and troubleshoot data pipeline performance and implement improvements as necessary. Document data processes and maintain clear communication with stakeholders regarding project status and challenges. Stay updated with the latest industry trends and technologies to continuously improve data engineering practices. Mandatory Skills: Strong proficiency in SQL for data manipulation and querying. Expertise in Python for data processing and automation. Hands-on experience with Databricks is a deal breaker. Familiarity with Azure Data Factory for orchestrating data workflows. Experience with Spark for big data processing. Knowledge of Delta Tables for efficient data storage and management. Understanding of Unity Catalog for data governance and management. Experience with Azure Data Lake for data storage solutions. Familiarity with Azure Synapse for data integration and analytics. Preferred Skills: Experience with data modeling and ETL processes. Knowledge of data warehousing concepts and best practices. Familiarity with machine learning concepts and tools. Experience with version control systems such as Git. Strong problem-solving skills and attention to detail.

Similar Jobs (35130)

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Engineering, SQL, Python, ETL, Data Pipelines, Apache Spark, Airflow, Data Warehousing, Data Modeling, Cloud Platforms, Kafka, dbt, Snowflake, AWS, Git

Data Platform Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Platform Engineering, Python, SQL, Apache Spark, Kafka, Kubernetes, Airflow, Cloud Platforms, Infrastructure as Code, Terraform, Data Lakes, Snowflake, Databricks, Data Governance, CI/CD

Data Platform Engineer

Accenture

Not disclosedBengaluru

Skills required:

Data Platform Engineering, Python, SQL, Apache Spark, Kafka, Kubernetes, Airflow, Cloud Platforms, Infrastructure as Code, Terraform, Data Lakes, Snowflake, Databricks, Data Governance, CI/CD

Custom Software Engineer

Accenture

Not disclosedBengaluru

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Custom Software Engineer

Accenture

Not disclosedBengaluru

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

View All 35130 Jobs