At the IROS conference in Vancouver, Li Feifei, a renowned computer vision expert and director of Stanford's AI Lab and Vision Lab, delivered a talk titled "A Quest for Visual Intelligence." This marked her first participation in the IROS robot-focused conference. Notably, a groundbreaking paper titled *Neural Task Programming: Learning to Generalize Across Hierarchical Tasks* was recently published on Arxiv. The lead author of this paper is from Stanford, with Li Feifei and her husband, Silvio Savarese, serving as co-advisors. This development highlights the growing convergence between artificial intelligence and robotics, especially in visual perception and understanding, and signals a future of closer interdisciplinary collaboration.
We introduced a novel robot learning framework called Neural Task Programming (NTP), which enables robots to learn from fewer demonstrations and integrate neural programming techniques. NTP recursively breaks down incoming tasks—such as video demonstrations—into hierarchical sub-tasks that are processed by a layered neural program. This program interacts with the environment through lower-level sub-programs that can be dynamically called. We tested our approach across three robotic manipulation tasks, where NTP demonstrated strong generalization capabilities in handling sequential and composite task structures. The results show that NTP excels at learning and adapting to variable-length, variable-topology, and unknown tasks.
**Background**
As human-machine collaboration becomes more central to modern robotics, robots are increasingly required to interact with humans over extended periods in dynamic environments like object sorting, assembly, and cleaning. Traditionally, robots operated in fixed settings, but today’s challenges involve making them adaptable to complex tasks, changing goals, and evolving surroundings.
Imagine an object classification task in a warehouse, involving steps like sorting, retrieving, and packaging. Each task can be broken into smaller actions—like grasping, moving, and placing—which form sub-tasks. When considering different objects, categories, and combinations, the complexity increases significantly. For example, sorting four types of items into four containers results in 256 possible combinations. In this paper, we aim to tackle two main challenges: (a) developing a learning strategy that transfers knowledge from one conceptual task to another, and (b) combining basic code modules that can interact with long-term environmental feedback.
**Neurological Programming (NTP) Principle**
The core idea behind NTP is cross-task learning and the reuse of shared representations. NTP interprets a task specification, which outlines the process and final goal, and translates it into a hierarchical neural program. It decodes the target object from the input specification, decomposes it into subtasks, and interacts with the environment until the goal is achieved. Each program takes in environment observations and task specifications, then outputs the next subroutine along with its corresponding subtask description.
As shown in the figure, given an input, task specification, and current environment, the NTP model predicts which subroutine to execute, passes the input to the next subtask, and determines whether the program should continue or terminate.
**Testing**
The research team evaluated NTP using simulated single-arm operations and real-world robot experiments. Tasks included stacking blocks, sorting targets, and cleaning a table.
The study had two main objectives: (I) learning multiple tasks within the same domain, and (II) achieving generalization through a single example.
As shown, the robot can learn from simulated demos—including images, videos, and VR inputs—and successfully complete block-stacking tasks.
NTP programming involves generating a policy induction program based on hierarchical task conditions and calling the robot API to perform actions.
The figure above shows a sample execution trace of NTP during a block-stacking task. The top-level program receives the overall task, predicts the next subroutine, and calls the appropriate robot action (e.g., Move_to(Blue), Grasp(Blue)). When the end-of-program (EOP) signal is triggered, the program stops and returns control to the caller.
If the environment changes—such as if a completed task is disrupted—the robot detects the change and re-performs the task.
Task structure changes include modifying completion conditions, varying subtask alignment, and increasing task length. As the number of tasks grows, NTP continues to improve performance for new tasks and goals.
All Accessories Of Laser Level
Green Laser Protective Glasses,Laser Protection Glasses,Laser Protection Eyewear,Laser protection
Guangdong Tumtec Communication Technology Co., Ltd , https://www.gdtumtec.com
