Over the last few years, the field of Artificial Intelligence has witnessed significant growth, largely fueled by the development of large-scale machine learning models. The foundational models are characterized by extensive training on diverse datasets that encompass various input modalities (e.g. images, text, audio, 3D data), showing excellent flexibility and effectiveness across a wide range of standard NLP and Computer Vision tasks. Such general-purpose solutions often reveal potentials that go beyond what their creators originally envisioned, motivating users to adopt these models for a broad spectrum of applications. Nevertheless, the knowledge embedded in these models may not be enough when the final goal goes beyond perception benchmarks. These considerations spark important questions that can only be answered with a collaborative dialogue between researchers developing these models (creators) and those employing them in downstream tasks (users). Each group brings a unique perspective that will be crucial in shaping the future of this technology.

The goal of this workshop is to identify and discuss strategies to assess both positive and negative (possibly unexpected) behaviors in the development and use of foundation models.

Particular attention will be given to applications that diverge significantly from the scenarios encountered during the training phase of foundational models. These include application-specific visual understanding, uncertainty evaluation, goal-conditioned reasoning, human habits learning, task and motion planning, scene navigation, vision-based manipulation, etc. Our purpose is to foster an open discussion between foundation model creators and users, targeting the analysis of the most pressing open questions for the two communities and fostering new fruitful collaborations.

A few examples are:

1. Should we always lean towards versatile generalist models, or are there circumstances where specialized task-specific models are still preferred?
2. What are the risk of blindly using foundation models?
3. What are the downstream tasks that are still challenging for current foundational models?
4. How the peculiarities of those tasks may guide an effective re-design of foundation models?
5. Do such difficulties arise from data-related issues, or are they rather caused by the learning objective and network design aspects?
6. What level of common sense and abstract reasoning is necessary for these models to operate effectively in real-world and possibly onboard robotic agents?
7. How can we properly benchmark these capabilities in current models?