Senior Big Data System Engineer

1 week ago Be among the first 25 applicantsAre you an experienced Big Data professional eager to tackle complex challenges in a dynamic banking environment? EPAM is seeking a dedicated Senior Big Data System Engineer to maintain, optimize and enhance distributed systems and data pipelines for one of our strategic clients in the financial sector. In this role, you will operate global data platform components, manage applications and integrate cutting-edge tools to deliver scalable and resilient solutions.We offer a hybrid work model with a mix of remote and on-site work at our client’s office in Zürich.ResponsibilitiesOperate and maintain core Global Data Platform components, including VM Servers, Kubernetes and Kafka, to ensure smooth system performance and functionalityManage data and analytics applications, such as Apache stack components, Dataiku, Collibra and other essential toolsAutomate infrastructure configurations, security components and CI/CD pipelines to drive efficiency and eliminate manual intervention in data pipelinesDevelop robust solutions to enhance platform resiliency, implement health checks, monitoring, alerting and self-recovery mechanisms for data operationsFocus on improving data pipeline quality by addressing accuracy, timeliness and recency in ELT/ETL executionEmbed Agile and DevSecOps best practices into delivery processes, ensuring iterative development and deployment of integrated solutionsCollaborate with stakeholders like enterprise security, digital engineering and cloud operations to align on effective solution architecturesKeep track of and evaluate emerging technologies across the Big Data ecosystem to deliver innovative features and capabilitiesRequirementsDemonstrated 5+ years of experience in building or designing fault-tolerant, large-scale distributed systems, showcasing an ability to manage complex infrastructure and operationsMastery of distributed data technologies such as data lakes, delta lakes, data meshes, data lakehouses and real-time streaming platformsExpert knowledge of tools like Kafka, Kubernetes and Spark, as well as formats like S3/Parquet for scalable data solutionsProficiency in Python and Java programming, or alternatives such as Scala/R, coupled with adeptness in Linux/Unix scriptingExperience in managing Docker (Harbor), VM setup/scaling, Kubernetes pod management and CI/CD pipelinesKnowledge of configuration management tools like Jinja templates, puppet scripts and best practices in firewall rules setupStrong understanding of DevOps practices for scalable deployment and automation strategiesFluency in English is essential, German language skills are considered an asset for collaborationFamiliarity with financial services and their unique data and regulatory challenges is an advantageWe offer5 weeks of vacationEPAM Employee Stock Purchase Plan (ESPP)Enhanced parental leaveExtended pension planDaily sickness allowance insuranceEmployee assistance programGlobal business travel medical and accident insuranceLearning and development opportunities including in-house training and coaching, professional certifications, over 22,000 courses on LinkedIn Learning Solutions and much more*All benefits and perks are subject to certain eligibility requirementsPlease note that any offers will be subject to appropriate background checks.We do not accept CVs from recruiting or staffing agencies.For this position, we are able to consider applications from the following:Swiss nationalsEU/EFTA nationalsThird-country nationals based in Switzerland with an appropriate work permitDisplaced people from Ukraine who are currently in Switzerland and hold, or have already applied for, S permitsSeniority level: Mid-Senior levelEmployment type: Full-timeJob function: Engineering, Information Technology, and Business DevelopmentIndustries: Software Development and IT Services and IT ConsultingNote: Referrals increase your chances of interviewing at EPAM Systems. #J-18808-Ljbffr