MTS, Research Engineer

About Us: At Fireworks, we’re building the future of generative AI infrastructure. Our platform delivers the highest-quality models with the fastest and most scalable inference in the industry. We’ve been independently benchmarked as the leader in LLM inference speed and are driving cutting-edge innovation through projects like our own function calling and multimodal models. Fireworks is a Series C company valued at $4 billion and backed by top investors including Benchmark, Sequoia, Lightspeed, Index, and Evantic. We’re an ambitious, collaborative team of builders, founded by veterans of Meta PyTorch and Google Vertex AI.About the Role We are looking for a Research Engineer to join our team, operating at the critical intersection of model research and training infrastructure. In this role, your time will be split between tackling open-ended research problems—such as designing novel architectures and improving algorithmic efficiency — and building the distributed training systems required to make those research breakthroughs a reality. You won't just be handed a paper to implement; you will be expected to reproduce state-of-the-art results from the literature, identify their limitations, and build the infrastructure needed to push beyond them. The most significant advances in deep learning require massive scale. We need engineers who are as comfortable reasoning about gradient descent and loss landscapes as they are about distributed systems, GPU cluster utilization, and data pipelines.   What You'll Do Conduct Open-Ended Research: Explore new model architectures, training objectives, and optimization techniques. Formulate hypotheses, design experiments, and iterate quickly based on empirical results. Reproduce and Extend State-of-the-Art: Implement and reproduce results from recent machine learning papers. Identify bottlenecks, propose improvements, and scale these methods to larger datasets and models. Build and Scale Training Infrastructure: Design, implement, and maintain high-performance, distributed machine learning systems. Optimize training loops, data loaders, and communication overhead across large GPU clusters. Bridge Science and Engineering: Translate abstract mathematical concepts and research ideas into robust, bug-free, and efficient code. Collaborate Cross-Functionally: Work closely with Research Scientists to unblock their experiments by providing tooling, optimizing code, and co-designing experiments that are hardware-aware. We Expect You To Have: Strong programming skills (Python, C++, or Rust) and a commitment to writing clean, maintainable code. Deep practical knowledge of machine learning frameworks (PyTorch, JAX, or TensorFlow). Experience working with large distributed systems and parallel computing (e.g., CUDA, NCCL, MPI). A strong foundation in linear algebra, calculus, probability, and statistics. A proven track record of implementing complex deep learning algorithms from scratch. Nice to Have: A Master’s or PhD...