Big Data Engineer

Description:

As a Software Engineer - Big Data, you will be involved and responsible for full ownership and driving key data platform initiatives and life cycle of the data management including data ingestion, data processing, data storage, querying system, cost reduction efforts towards delivering product features to internal and external Motive customers.

We are looking for strong engineers to grow our Data Platform team. This team is responsible for driving the Motive’s Data Platform vision. The team works in three areas: Build scalable systems and services for data ingestion, access, processing and query to enable data driven product features. Collaborate and work closely with the various stakeholders and our backend product teams to improve and add features to the platform.

What You’ll Do
 

• Design and develop scalable distributed systems and frameworks for data management
• Focus on addressing fault-tolerance and high availability issues, and work on scaling ingestion pipelines, improving and adding features to ETL framework while maintaining SLAs on performance, reliability, and system availability.
• Collaborate with engineers across teams to identify and deliver cross-functional features
• Participate in all aspects of the software development life cycle, from design to implementation and delivery.
   

What We’re Looking For
 

• 3+ years Hands-on software engineering experience
• Backend programming skills including multi-threading, concurrency, etc. and proficient in Python
• Strong CS fundamentals including data structures, algorithms, and distributed systems
• Experience in designing, implementing, and operating highly scalable software systems and services
• Experience building systems using technologies like Apache Kafka, Apache Spark, Airflow, Kubernetes
• Experience with service-oriented architectures
• Excellent troubleshooting skills and track record of implementing creative solutions
• Hands on experience with containerized platforms like Docker and Kubernetes
• BS in Computer Science or a related field; Masters preferred
• Excellent verbal and written skills. You collaborate effectively with other teams and communicate clearly about your work.