Senior Software Engineer - Data Infrastructure

Sarvam AI

2-5 YrsNot disclosedBengaluru (On-site)
Posted: 1d agoOpening: 1

Skills Required

Key skills are marked withstar
  • Key skillSoftware Engineering
  • Key skillSystem Design
  • Key skillDistributed Systems
  • Data Structures & Algorithms
  • Java
  • Python
  • Go
  • Microservices
  • REST APIs
  • SQL
  • Cloud Platforms
  • Docker
  • Kubernetes
  • CI/CD
  • Code Review

Job Description

The Role

Sarvam's products produce events, model traces, evals and billing data, and need to read that data back. The Data Platform team builds and runs the central data infrastructure every product uses: ingestion, streaming, the analytical warehouse, transformations, access control and audit.

We are hiring a Software Engineer / Senior Software Engineer (2–5 years of experience) to own this platform end to end. You will design, build and run systems that other teams depend on in production, and make real decisions on database internals, streaming and platform design. Stack: Go/Python, ClickHouse Cloud, Kafka, Avro, Airflow, dbt, Kubernetes, Terraform, AWS.


What You'll Own & Do

  • Build and run the ingestion path: the service every product sends events to, Kafka behind it, and the pipes that land data in ClickHouse within seconds. Own schema validation, rate limits, dead-letter handling and the client SDKs.

  • Own the warehouse: ClickHouse Cloud. Design tables for the read path (sorting keys, partitions, materialized views, TTL), separate compute per workload, set per-user limits and per-tenant isolation, and attribute cost to the team that spent it.

  • Build reliable data pipelines and solve data problems: dbt models and Airflow DAGs with the BI engineers, CDC from product databases, sync and backfill jobs that are safe to re-run, and the data problems product teams bring you, such as a late source, a number that doesn't match or a query that has slowed down.

  • Make the platform self-serve: a registry of every dataset with its owner, schema, freshness and access; provisioning and permissions as code.

  • Run it in production: everything as Terraform and Helm with metrics, traces and alerts; be on call; handle incidents end to end, write the RCA and fix the root cause.

  • Grow the platform: new products keep arriving. You will help decide what the platform should do next and build it, whether that is a new source or sink, or a new way for teams to use the data.

What We're Looking For

  • 2–5 years of experience building, shipping and running large-scale, data-intensive systems in production, ideally at a startup where you owned things and were on call for them.

  • Deep expertise in at least one of: an OLAP database (ClickHouse preferred; Druid, BigQuery or Redshift also fine) or Kafka / event streaming. You understand how a columnar store writes, merges and scans data, or how a partitioned log gives ordering and delivery guarantees.

  • Experience with large-scale messaging systems like Kafka or RabbitMQ: partitions, consumer groups, delivery guarantees, backpressure.

  • Good at Go, or good at Python.

  • You know the usual data-system problems: schema evolution, idempotency, backpressure, partitioning, retention and TTL, CDC.

  • You have built or run at least one of: an event pipeline, a transformation layer (dbt, Spark or SQL on Airflow), a provisioning or control-plane service, or a multi-tenant data service with access control.

  • Comfortable with AWS, Kubernetes and Terraform in production.

  • You test before shipping, debug with metrics and traces, write RCAs, and document what you build.

  • Passion for data infrastructure and for enabling others by making their data easier to access.

Nice to Have

  • ClickHouse in production: MergeTree, materialized views, projections, quotas, settings profiles, row policies, ClickHouse Cloud.

  • Kafka in depth: MSK, Kafka Connect, schema registry (Glue or Confluent), Avro.

  • dbt and Airflow at scale.

  • Audit or security data systems: append-only logs, integrity checks, tenant isolation.

  • OpenTelemetry, LLM observability (Langfuse or similar), eval pipelines.

  • Built an internal platform that other teams adopted.

Similar Jobs (35112)

Software Dev Engineer I, L4

Amazon

Not disclosedBengaluru

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Software Development Engineer , UTR - Pick Stage

Amazon

Not disclosedGurugram

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Software Development Engineer - Agentic Data Infrastructure, Partner Data

Amazon

Not disclosedChennai

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Software Dev Engineer II , Last Mile Delivery Prdct&Tech

Amazon

Not disclosedHyderabad

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Software Development Engineer, Last Mile Platform Services, SDE

Amazon

Not disclosedGurugram

Skills required:

Programming, Data Structures & Algorithms, System Design, Object-Oriented Programming, Java, Python, JavaScript, SQL, REST APIs, Git, Unit Testing, Cloud Platforms, Docker, CI/CD, Debugging

Lead Software Engineer (Level 5) | BackEnd (Gurugram based)

Agoda

Not disclosedGurugram

Skills required:

Technical Leadership, System Design, Software Architecture, Code Review, Mentoring, Programming, Distributed Systems, Microservices, Cloud Platforms, CI/CD, Agile Methodologies, Design Patterns, Stakeholder Management, Estimation, Git

Sr. Business Intelligence Engineer, AOP (Analytics Operations and

Amazon

Not disclosedBengaluru

Skills required:

Machine Learning, Python, Deep Learning, PyTorch, TensorFlow, scikit-learn, MLOps, Model Deployment, Feature Engineering, SQL, Docker, Kubernetes, Cloud Platforms, Data Pipelines, Model Evaluation

Staff Software Engineer (Backend)

6sense

Not disclosedBengaluru

Skills required:

Software Architecture, System Design, Distributed Systems, Technical Leadership, Scalability, Programming, Cloud Architecture, Microservices, Performance Optimization, Cross-Team Collaboration, Mentoring, Code Review, Technology Strategy, Security Best Practices

Lead Engineer

AECOM

Not disclosedChennai

Skills required:

Technical Leadership, System Design, Software Architecture, Code Review, Mentoring, Programming, Agile Methodologies, Cloud Platforms, CI/CD, Stakeholder Management, Estimation, Problem Solving, Cross-Functional Collaboration

Lead Engineer

AECOM

Not disclosedChennai

Skills required:

Technical Leadership, System Design, Software Architecture, Code Review, Mentoring, Programming, Agile Methodologies, Cloud Platforms, CI/CD, Stakeholder Management, Estimation, Problem Solving, Cross-Functional Collaboration

View All 35112 Jobs