;word-spacing:0px”> Perception, Action, Learning: from Metric-Semantic Scene Understanding to High-level Task Execution

International Conference on Robotics and Automation

Website: https://mit-spark.github.io/PAL-ICRA2020/

Submission Link:  https://easychair.org/conferences/?conf=pal2020icraworkshop

Submission Deadline: April 15, 2020

Challenge Submission Deadline: May 20, 2020

OVERVIEW:

—————-

This workshop brings together researchers from robotics, computer vision, and machine learning to examine challenges and opportunities emerging at the boundary between spatial perception and high-level task execution. Recent years have seen a growing interest towards metric-semantic understanding, which consists in building a semantically annotated (or object-oriented) model of the environment. On the other hand, researchers have been looking at high-level task execution using modern tools from reinforcement learning and traditional decision-making. The combination of these research efforts in spatial perception and task execution has the potential to enable applications such as visual question-answering, object search and retrieval, and provides a more intuitive interaction with the user. This workshop creates an exchange opportunity to connect researchers working in metric-semantic perception and task execution. In particular, the workshop will bring forward the latest breakthroughs and cutting-edge research in the two research areas. Besides the usual mix of invited talks and poster presentations, the workshop involves two interactive activities. First, we will provide a hands-on tutorial on a state-of-the-art library for metric-semantic reconstruction (Kimera). Second, we will organize the GOSEEK challenge (details below), in conjunction with the release of a photo-realistic Unity-based simulator, where participants will need to combine perception and decision-making to find objects in a complex indoor environment.

– The workshop will include keynote presentations from established researchers in robotics, machine learning, computer vision, robot perception.

– There will be two spotlight talks and two poster sessions highlighting contributed papers throughout the day.

– The winner of the challenge will give an invited keynote presentation  (and take home a monetary prize).

The workshop is endorsed by the IEEE RAS Technical Committee for Computer & Robot Vision.

CHALLENGE AND AWARDS:

————–

We are organizing the GOSEEK reinforcement learning challenge. The challenge consists in creating an RL agent that combines advanced perception (provided by Kimera) and high-level decision-making to search for objects placed within complex indoor environments from a Unity-based simulator. Simply put: like PACMAN, but in a realistic scene and with realistic perception capabilities. Several data modalities are provided from both the simulator ground truth and the perception pipeline (e.g., images, depth, agent location) to enable the participants to focus on the RL/search aspects. The contest will be hosted on the EvalAI platform, where participants will submit solutions, via docker containers, for scoring. The winner of the competition will receive a monetary prize ($1000) and will give a keynote presentation at the workshop.

Tentative timeline:

– February 15: Challenge website is online: https://github.com/MIT-TESSE/goseek-challenge

– April 15: EvalAI platform ready for official submission and scored runs!

– May 20: Final deadline to submit your RL agent! 

SUBMISSIONS:

———————

Participants are invited to submit an extended abstract or short papers (up to 4 pages in ICRA format) focusing on novel advances in spatial perception, reinforcement learning, and at the boundary between these research areas.
Topics of interest include but are not limited to:

– Novel algorithms for spatial perception that combine geometry, semantics, and physics, and allow reasoning over spatial, semantic, and temporal aspects;

– Learning techniques that can produce cognitive representations directly from complex sensory inputs;

– Approaches that combine learning-based techniques with geometric and model-based estimation methods;

– Novel transfer learning and meta-learning methods for reinforcement learning;

– Novel RL approaches that leverage domain knowledge and existing (model-free and model-based) methods for perception and planning; and

– Position papers and unconventional ideas on how to reach human-level performance in robot perception and task-execution. 

Contributed papers will be reviewed by the organizers and a program committee of invited reviewers. Accepted papers will be published on the workshop website and will be featured in spotlight presentations and poster sessions. 

Submission link: https://easychair.org/conferences/?conf=pal2020icraworkshop

IMPORTANT DATES:

—————————-

– Submission Deadline: April 15, 2020

– Notification of Acceptance: May 10, 2020

– Final deadline for GOSEEK challenge : May 20, 2020 

– Workshop Date: May 31, 2020 (TBC)

ORGANIZING COMMITTEE:

————————————-

– Luca Carlone, Massachusetts Institute of Technology

– Dan Griffith, Massachusetts Institute of Technology Lincoln Laboratory

– Sanjeev Mohindra, Massachusetts Institute of Technology Lincoln Laboratory

FURTHER INFORMATION:

———————————–

Please send any questions to Luca Carlone (lcarlone@mit.edu). Please include "PAL ICRA 2020 Workshop" in the subject of the email.

————————————————————————————

Luca Carlone

Charles Stark Draper Assistant Professor

Laboratory for Information and Decision Systems (LIDS)

Massachusetts Institute of Technology (MIT)

office: 32 Vassar St., Cambridge, MA 02139, Room: 31-243

web: http://www.lucacarlone.com/

————————————————————————————