Data Engineer - 1

Bengaluru, Karnataka, India | Engineering | Full-time | COVID-19 remote

Apply

Roles and Responsibilities

  • ¬†Build highly scalable, available, fault-tolerant data processing systems using Kafka, Spark, Bigquery, Flink, Airflow, and other big data technologies. These systems should handle batch and real-time data processing over terabytes of data ingested every day and a petabyte-sized data warehouse.
  • Collaborate with our Engineering and Data Science teams on building, maintaining, and monitoring the data infrastructure.
  • Participate in architecture discussions, influence product roadmap, and take ownership of projects.
  • Experience extending and implementing core functionality and libraries in data processing platforms (like Flink, Spark/Spark SQL, Storm Bolts, etc.)
  • Willingness to learn new languages and methodologies.

Qualifications

  • Expertise in architecting and building large-scale data processing systems using Big Data technologies like Hadoop, Hbase, Spark, Storm, Kafka, Druid, Flink, Airflow, etc.
  • 1-2 Years of overall experience in building highly scalable systems.
  • Experience in building highly scalable solutions in data storage, real-time analysis, and reporting for multi-terabyte data sets.
  • Integrate data from multiple data sources, develop cross-platform ETL processes
  • Fast learner and eager to absorb new emerging technologies
  • Experience with any programming language ( Java, python, go..etc) and Infrastructure as Code tools (e.g. Terraform, Ansible, Crossplane, etc.)
  • Experience with DevOps/SRE practices/principles
  • Database performance concepts like indices, segmentation, projections, clustering and partitions