Big Data Engineer (Python)
Ogłoszenie wygasło. Aplikowanie wyłączone.
Replatforming from AWS to GCP. Project goal is to build an initial version of Infrastructure using GCP that will host specific services and applications that will be migrated from AWS during the next project phases. Complete the initial research, identify the first applications to be migrated on later program phases, and build a migration plan for selected applications or services. Scope includes establishing services for ingestion of data into GCS storage layer, Data transformation using Spark, Implementation of Cloud Composer (managed Airflow), Google Data Catalog
• Establish scalable, efficient, automated processes for data analyses, model development, validation and implementation
• Work closely with data scientists and analysts to create and deploy new features
• Write efficient and well-organized software to ship products in an iterative, continual-release environment
• Monitor and plan out core infrastructure enhancements
• Contribute to and promote good software engineering practices across the team
• Mentor and educate team members to adopt best practices in writing and maintaining production code
• Communicate clearly and effectively to technical and non-technical audiences
• Actively contribute to and re-use community best practices
• Embody the values and passions that characterize Levi Strauss & Co., with empathy to engage with colleagues from a wide range of backgrounds
• University or advanced degree in engineering, computer science, mathematics, or a related field
• Strong experience working with a variety of relational SQL and NoSQL databases
• Strong experience working with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with at least one cloud provider solution (AWS, GCP, Azure)
• Strong experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Ability to work in Linux environment
• Experience working with APIs
• Strong knowledge of data pipeline and workflow management tools
• Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation
• Experience creating ETL processes that prepare data for consumption appropriately
• Experience in setting up, maintaining and optimizing databases for production usage in reporting, analysis and ML applications
• Working in a collaborative environment and interacting effectively with technical and non-technical team members equally well
• Relevant working experience with Docker and Kubernetes preferred
• Ability to work with ML frameworks preferred
• a B2B contract or a permanent contract (UOP);
• a benefit basket and an insurance program;
• flexible working hours (the contract presumes 40 hours a week, but we are flexible);
• working 100% remotely, from the office, or in a hybrid model.