VLM Engineer

Technology Innovation Institute

As part of TII s Artificial Intelligence Research Center, the Extreme-Scale Language Model team is developing and implementing innovative deep learning technologies with a broad range of applications, from Natural Language Processing to Perception and Vision. Our team has developed the Falcon models and is now planning to continue our journey into cutting-edge applied research in the topic of large language models.

Key Responsibilities:

• Vision Model Ablation Studies: Conduct comprehensive ablation studies on vision models to assess the impact of various components and configurations. Collaborate with researchers to analyze and report on the effectiveness of different model architectures and settings.
• Data Ablation Research: Partner with team members to perform data ablation studies, identifying optimal data types and structures for training vision-language models. Also, analyze the impact of different data inputs on model performance, particularly focusing on vision-language alignment.
• Model Evaluation: Develop and implement robust evaluation protocols for vision language models. Assess model performance across diverse benchmarks and real-world scenarios.
• Model Training and Optimization: Engage in model training, with an emphasis on integrating LLMs with vision models like CLIP.