Company: AI71
Role: LLM Engineer
Location: Abu Dhabi, UAE
About Us:
AI71 is an applied research team dedicated to creating helpful and responsible AI agents for knowledge workers.
Working closely with our industry partners, our cross-functional teams of AI experts build products grounded in the cutting-edge research of our colleagues from the Technology Innovation Institute (TII).
Job Description:
As a Senior LLM Engineer, you will be responsible for the end-to-end development, optimization, and deployment of large language models. You'll work on challenging problems at the intersection of deep learning, natural language processing, and distributed computing.
What You'll Do:
- Analyze large and complex datasets to extract meaningful insights and inform data-driven decision-making.
- Develop, train, and deploy predictive models to enhance the capabilities of our AI solutions.
- Collaborate with cross-functional teams to understand business objectives and translate them into actionable data science tasks.
- Design and implement advanced LLM architectures, including transformer-based models and their variants
- Develop novel attention mechanisms and positional encoding schemes
- Experiment with model scaling techniques and efficient architectures (e.g., MoE, sparse transformers)
- Continuously evaluate and improve existing models based on real-world performance and evolving business needs.
- Implement and optimize distributed training pipelines for large-scale models
- Develop strategies for efficient fine-tuning, including parameter-efficient techniques (e.g., LoRA, prefix tuning)
- Apply advanced optimization techniques such as mixed-precision training and gradient accumulation
- Optimize models for inference, including quantization and pruning techniques
- Implement efficient serving solutions for real-time inference
- Develop strategies for model compression and knowledge distillation
- Develop task-specific algorithms for applications such as text classification, named entity recognition, and question-answering
- Work with MLOps teams to design and maintain training and serving infrastructure
What You'll Bring:
- 5+ years of experience in deep learning and NLP, with a focus on large language models
- Master's or Ph.D. in Data Science, Statistics, Computer Science, or a related field.
- Expert-level proficiency in Python and at least one deep learning framework (PyTorch, TensorFlow, or JAX)
- Strong understanding of transformer architectures, attention mechanisms, and recent advancements in LLMs
- Experience with distributed training frameworks (e.g., DeepSpeed, Megatron-LM)
- Proficiency in optimizing model performance using techniques like mixed-precision training, gradient checkpointing, and model parallelism
- Understanding of NLP algorithms such as tokenization, parsing, and semantic analysis
- Experience with sequence-to-sequence models and self-supervised learning techniques
- Experience with both SQL and NoSQL databases for managing training data and model artifacts
Why AI71:
- Proven performance of our large language models
- Strong traction and adoption from the open-source community
- Secured proprietary data to build specialized distinctive models.
- Locked large compute power to support our roadmap.
- Signed anchor clients, to develop POCs and demonstrate our solutions.
