Senior Data Engineer
Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It’s the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever.A Little About UsThe Mail Analytics Infrastructure & Data Engineering team at Yahoo is the backbone of Yahoo Mail’s success. We build mission-critical data lakehouses, high-performance pipelines, and the analytics frameworks that power one of the world's largest communication platforms. We are currently pushing the boundaries of our data ecosystem to harness massive datasets, directly fueling the explosive growth of Yahoo’s AI-driven capabilities.A Lot About YouYou are a Senior Data Engineer who thrives on complexity and scale. You don't just build pipelines; you define the data ontology for the entire Mail organization. You are a strategic thinker who can influence event instrumentation at the source while collaborating with Data Science and ML teams to deliver high-impact analytics. ResponsibilitiesArchitect: Partner with Data Science, Product, and Engineering to define the global data ontology for Yahoo Mail and lead the technical roadmap for core datasets.Build Scalable Systems: Design, build, and maintain high-reliability batch and streaming data pipelines that populate our mission-critical data lakehouse.Innovate Tooling: Develop automated frameworks and self-service tools that streamline how users interact with data products across the company. Leveraging autonomous AI agents and AI Tooling. Data Governance: Establish standard methodologies for data operations, lifecycle management, and strict SLA management for all datasets within your ownership area.Optimize Performance: Improve existing large-scale data infrastructures by applying advanced algorithmic concepts to optimize code and underlying data system stacks.Collaborate: Act as a data consultant for complex cross-functional projects, ensuring integrated solutions across Mail engineering teams.QualificationsEducation: BS/MS in Computer Science, Engineering, or a relevant technical field.Experience: 6+ years of experience building scalable ETL/ELT pipelines using industry-standard orchestration (Airflow, Composer, or Oozie).Technical Mastery: Deep expertise in SQL, PySpark, or Scala.Scale: Proven track record of managing Multi-Terabyte/Petabyte datasets and solving large-scale challenges (e.g., skew mitigation, data sketches, and accumulation patterns).Cloud & DevOps: Professional experience with at least one major cloud provider (GCP, AWS, or Azure) and a strong command of GitOps workflows (CI/CD, PRs).Compliance: Experience working within GDPR and other data privacy frameworks.Soft Skills: Exceptional communication skills with the ability to prioritize tasks in a high-pressure, fast-paced environment.Preferred QualificationsGCP Expertise: 3+ years of experience with Google...