Senior Data Engineer

Zensar Technologies

Not disclosedBengaluru (On-site)
Posted: 13d agoOpening: 1

Skills Required

Key skills are marked withstar
  • Key skillData Engineering
  • Key skillSQL
  • Key skillPython
  • Apache Spark
  • Data Pipelines
  • Airflow
  • Kafka
  • Data Modeling
  • Data Warehousing
  • Snowflake
  • Databricks
  • Cloud Platforms
  • dbt
  • Data Quality
  • CI/CD

Job Description

We're looking for a Senior Data Engineer with strong data architecture and system design skills to design, build, and own end-to-end production pipelines powering analytics and decision-making across the org. The role centers on deep Python + SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain. Internal candidates familiar with Uber's data platform stack (Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder, OneETL) will be prioritized.

● Own data architecture and system design decisions for pipelines and data models - grain, partitioning, schema evolution, and scalability tradeoffs. ● Take end-to-end ownership of pipelines in production: design, build, deploy, monitor, and operate. ● Design and build transaction tables (append-only, immutable event/transaction-grain fact tables) alongside standard fact/dimension models using medallion (bronze/silver/gold) architecture. ● Build and support pipelines for finance, risk, or compliance use cases where accuracy, auditability, and data lineage are critical. ● Implement data quality and auditability controls: validation checks, reconciliation logic, anomaly detection, and audit trails for every pipeline you own. ● Write complex, cross-dialect SQL (MySQL + PostgreSQL): window functions, multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and date/timestamp handling. ● Build idempotent reload patterns (DELETE+INSERT), UNION ALL/set operations, and templated (Jinja-style) SQL including handling for late-arriving/corrected transaction records without double-counting. ● Orchestrate Airflow-style DAGs (Pipeline/BaseTask, ExternalTaskSensor for cross-pipeline deps) with secure credential handling via uSecret. ● Build Python ETL tooling: pandas (CSV ↔ DB), SQLAlchemy + raw drivers (MySQLdb, psycopg2), type hints/dataclasses, class-based pipeline design. ● Integrate with Google Drive/Sheets APIs; build YAML-driven pipeline configs; handle Piper staging/user_staging quirks. ● Own CI/CD for pipelines via standard Git workflows. ● Use Uber's internal stack day-to-day: OneETL for pipeline authoring, Piper/uWorc for scheduling, Databook for lineage, uSecret for credentials, DSW notebooks for CSV/table work, SourceGraph for code search, Query Builder for ad hoc SQL.

● 6+ years in data engineering. ● Strong data architecture and system design skills to make sound tradeoffs on modeling, partitioning, and scalability, not just write queries. ● Demonstrated end-to-end production ownership of data pipelines from design through deployment, monitoring, and operational support. ● Expert-level Python + SQL as the primary toolset. Hive/Hadoop is a nice-to-have only. ● Robust data quality and auditability practices - validation, reconciliation, anomaly detection, audit trails. ● Experience working with finance, risk, or compliance data - comfort with the heightened accuracy, auditability, and data-lineage requirements typical of these domains. ● Proven experience designing and building transaction/fact tables. ● Expert SQL (window functions, CTEs, CASE logic, cross-dialect MySQL/PostgreSQL). ● Strong Python (pandas, SQLAlchemy, type hints/dataclasses, class-based pipelines). ● Airflow-like orchestration experience (Piper/uWorc a plus). ● Solid data modeling fundamentals (fact/dim, normalization, medallion architecture). ● YAML-based config experience; standard CI/CD and Git workflows.

Similar Jobs (35139)

Software Engineer - SnowFlake

YASH Technologies

Not disclosedBengaluru

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Full Stack Google AI Engineer

Hindustan Unilever

Not disclosedBengaluru

Skills required:

JavaScript, TypeScript, React, Node.js, Python, REST APIs, GraphQL, SQL, NoSQL, System Design, Microservices, Docker, Cloud Platforms, CI/CD, Unit Testing

Sr. Software Engineer-LIMS

YASH Technologies

Not disclosedPune

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Intermediate Applications Developer

UPS

Not disclosedChennai

Skills required:

Cloud Platforms, AWS, Microsoft Azure, Python, Java, Node.js, Serverless Architecture, Microservices, REST APIs, Docker, Kubernetes, Infrastructure as Code, CI/CD, Cloud Databases, Git

Intermediate Applications Developer

UPS

Not disclosedChennai

Skills required:

Cloud Platforms, AWS, Microsoft Azure, Python, Java, Node.js, Serverless Architecture, Microservices, REST APIs, Docker, Kubernetes, Infrastructure as Code, CI/CD, Cloud Databases, Git

Sr. Data Analyst - Qliksense

YASH Technologies

Not disclosedBengaluru

Skills required:

Data Analysis, SQL, Excel, Python, Power BI, Tableau, Data Visualization, Statistics, Data Cleaning, Dashboards, Reporting, Pandas, Google Sheets, Storytelling with Data, Stakeholder Management

SOC Automation & DevOps Manager

Hindustan Unilever

Not disclosedBengaluru

Skills required:

DevOps, Engineering Management, Team Leadership, CI/CD, Cloud Platforms, Site Reliability Engineering, Infrastructure as Code, Kubernetes, Incident Management, Release Management, Hiring, Budget Management, Stakeholder Management, Process Improvement, DevSecOps

Sr. Data Analyst - MS Excel

YASH Technologies

Not disclosedIndore

Skills required:

Data Analysis, SQL, Excel, Python, Power BI, Tableau, Data Visualization, Statistics, Data Cleaning, Dashboards, Reporting, Pandas, Google Sheets, Storytelling with Data, Stakeholder Management

Tech Lead - Mainframe

YASH Technologies

Not disclosedIndore

Skills required:

Technical Leadership, System Design, Software Architecture, Code Review, Mentoring, Programming, Agile Methodologies, Sprint Planning, Stakeholder Management, Cloud Platforms, CI/CD, Estimation, Cross-Functional Collaboration, Problem Solving

Business Analyst - Forecasting

YASH Technologies

Not disclosedIndore

Skills required:

Sales Forecasting, Demand Forecasting, Time Series Analysis, Excel, SQL, Python, Statistical Modeling, Data Analysis, Pipeline Analytics, Salesforce, Tableau, Variance Analysis, Reporting, Stakeholder Communication

View All 35139 Jobs