Internship : Exploring Near Real-Time Data Processing at Scale with Apache Flink and Apache Spark

vor 21 Stunden

Eastern Switzerland ELCA Vollzeit

DescriptionDescription-This internship offers an in-depth exploration and comparison of two leading stream processing frameworks — Apache Flink and Apache Spark — within the context of near real-time data processing.The intern will gain hands-on experience designing and implementing near real-time data pipelines using Apache Kafka as the messaging backbone, and processing data streams with both Flink and Spark. The project will include the development of practical use cases involving near real-world data sources, such as event streams from databases or web activity logs.The final deliverable will consist of performance benchmarks, scalability assessments, and recommendations outlining the strengths and limitations of each framework across different data streaming scenarios.ObjectivesUnderstand the fundamental concepts of Kafka, Flink, and Spark, including their architecture and use cases.Implement a pipeline to process streaming data from a single source using Kafka and Flink/Spark, gain insights about the technologies and test optimizations.Build a second pipeline with a more complex setup:Database → Debezium → Kafka → Flink/Spark → Operational and Analytical Queries.Handle multiple tables and implement watermarking to ensure synchronized data processing.Compare Flink and Spark based on performance, ease of use, and suitability for specific use cases.Document findings and propose guidelines for choosing between the two frameworks.Our offer• A dynamic work and collaborative environment with a highly motivated multi-cultural and international sites team• The chance to make a difference in peoples' life by building innovative solutions• Various internal coding events (Hackathon, Brownbags), see our technical blog• Monthly After-Works organized per locationsSkills requiredCore Skills:Basics of data engineering and distributed systems.Knowledge of SQL and database concepts (e.g., relational databases, transactions).Understanding of streaming concepts and data pipelines (e.g. Kafka, Flink, Spark).Technical Skills:Familiarity with Docker and containerized environments.Knowledge of Kafka and concepts like producers, consumers, topics, and partitions.Basic programming skills in Python, Java, or Scala.Understanding of event-driven architectures and CDC tools (Debezium is a plus).Exposure to cloud platforms (e.g., AWS, Azure, or GCP) is an advantage.Other Skills:Analytical thinking and problem-solving skills.Ability to learn new tools and technologies quickly.Interest in benchmarking and performance evaluation.This internship starts in February 2026. Applications must include your most recent academic transcripts (grades); applications without transcripts will not be considered.

  • Senior Data Engineer

    vor 21 Stunden

    Eastern Switzerland ELCA Vollzeit

    DescriptionFor the Data Analytics and AI business line, we are looking for a motivated Data Engineer with experience in implementing complex distributed IT environments that process or integrate large volumes of data. This requires competencies in big data technologies such as Apache Spark, MPP, relational databases, and data architecture. The ideal...

  • Applied Data Engineer

    vor 21 Stunden

    Switzerland Datasys Vollzeit

    Datasys builds high-scale, AI-powered data platforms that helps enterprises leverage deep consumer and business intelligence. If you love large-scale data engineering and innovation, we encourage you to apply.Our environment best suits those who:Excel in hard work and a high-velocity atmosphere. Value exceptional performance and continuous improvement. We...

  • Internship : Real-Time Technology Radar

    vor 21 Stunden

    Eastern Switzerland ELCA Vollzeit

    DescriptionDescriptionThis internship aims to create a real-time Technology Radar that provides a clear view of the organization's current technology landscape. By mapping technologies used across projects and comparing them with industry standards and emerging trends, the radar will support strategic decision-making, highlight opportunities for innovation,...

  • Technical Project Manager

    vor 21 Stunden

    Eastern Switzerland Avanda Solutions Vollzeit

    We are seeking a highly skilled Technical Project Manager to lead the delivery of our client Applicative Platform, built on Databricks and Apache Spark. This role demands both technical fluency and project leadership, ideally with a background in cloud -native data platforms and Agile delivery. You will collaborate closely with platform engineers,...

  • Real Time Trader Associate

    vor 21 Stunden

    Eastern Switzerland aps Vollzeit

    Arizona Public Service generates reliable, affordable and clean energy for 2.7 million Arizonans. As the state's largest and longest-serving energy provider, our more than 6,000 dedicated employees power our vision of creating a sustainable energy future for Arizona.SummaryThe Real Time Trading Associate will assist the more senior Real-Time Trader to ensure...

  • Real Time Trader

    vor 21 Stunden

    Eastern Switzerland aps Vollzeit

    Arizona Public Service generates reliable, affordable and clean energy for 2.7 million Arizonans. As the state's largest and longest-serving energy provider, our more than 6,000 dedicated employees power our vision of creating a sustainable energy future for Arizona. SummaryThe Real Time Trader ensures that APS customers are provided the most economical...

  • Internship : Building the Framework for Scalable Intelligence

    vor 21 Stunden

    Eastern Switzerland ELCA Vollzeit

    DescriptionDescriptionBringing AI projects from proof-of-concept to production is a complex journey. It involves managing experiments, ensuring reproducibility, and deploying robust pipelines. With GenAI, the challenges shift, as prompt management, evaluation of non-deterministic outputs, document preparation, and versioning become central concerns.Indeed,...

  • Founding AI Engineer

    vor 21 Stunden

    Eastern Switzerland Bjak Vollzeit

    Transform language models into real-world, high-impact product experiences.A1 is a self-funded AI group, operating in full stealth. We're building a new global consumer AI application focused on an important but underexplored use case.You will shape the core technical direction of A1 - model selection, training strategy, infrastructure, and long-term...

  • Product Manager, Data and AI 80-100

    vor 21 Stunden

    Eastern Switzerland RepRisk AG Vollzeit

    Company DescriptionAbout UsRepRisk is the world's most respected Data as a Service (DaaS) company for reputational risks and responsible business conduct. Our mission is to provide transparency on business conduct risks to drive positive change. Combining advanced AI with deep human expertise, and a proven methodology at the core, RepRisk's solutions bring...

  • CAD AI Software Engineer

    vor 21 Stunden

    Eastern Switzerland Imnoo AG Vollzeit

    Job descriptionDo you feel like you're just coding toy problems without making a real impact? Join us for a fulfilling career as a Software Engineer specializing in Big Data Pipelines and ML Automation, where you'll tackle concretely defined, real-world challenges. At Imnoo, you'll make a significant impact on automating manufacturing processes through...