The most accurate definition of this role is a blend of data engineering & Python development.
Responsibilities:
- End-to-end infrastructure development of Python microservices
- Optimizing process run-time and solving problems in a scalable manner
- Taking a major part in designing and implementing complex high scale systems, data pipelines and algorithms using a variety of technologies
- Working closely with the data science team, to define and implement an automated training pipeline for machine learning models
- Being in charge of development and maintenance of data processing engine
- Provide data engineering capabilities and frameworks to other company team
Requirements:
- BSc. in Computer Science, a related technical field, or equivalent experience
- Deep understanding of Python
- 4+ years of experience as a software developer
- At least 3+ years experience with Python programming language
- Experience with building scalable, high-performance, data-oriented production systems
- Experience with Micro services architecture and communication architecture (MQ)
- Experience with Cloud environments AWS/GCP/Azure/Oracle
- Experience with Mongo – advantage
- Familiarity with Big Data technologies such as Hadoop, Spark, HDFS, Airflow, HBase, etc. – Advantage
Technologies:
- Python, Node.JS
- Kubernetes, Docker, Jenkins, RabbitMQ, Redis
- AWS (EC2, S3, Lambda, Streaming, EMR)
Reviews
There are no reviews yet.