Memories.ai creates a standardized egocentric video pipeline

Memories.ai is building a data pipeline for robotics companies that combines standardized video collection with AI-powered annotations to turn everyday self-centered footage into training data for robots.

The San Francisco-based company is developing software and artificial intelligence to help robots understand the physical world. It builds video understanding models and data infrastructures, collects and annotates first-person (egocentric) videos to provide high-quality training data to robotics companies.

The long-term goal is to become a data infrastructure partner for robotics and physical AI, not just a data provider.

Memories.ai builds a platform on video data

Memories.ai said its video data infrastructure platform is designed for physical AI, robotics, world models, vision-language-action (VLA) models and humanoids. It provides a curated video data lake with egocentric videos, along with custom annotation services and dataset tools to train robot learning models.

The company obtains egocentric video through data collectors and partners, providing them with standardized capture hardware, described as a modular camera setup, and a uniform process for collecting footage consistently.

Memories.ai then processes and annotates the data with its models and makes it available through its data infrastructure. The company has published multiple pieces of its research on its GitHub repository. He has also published at ICLR, NeurIPS, and EMNLP events.

NVIDIA Cosmos helps realize the vision of physical AI

“Our vision right now is to accelerate physical intelligence, and our vision is to really be on the cutting edge of being part of the robotics game,” Shawn Shen, co-founder and CEO of Memories.ai, recently said The robot report. “We really bring robotics into the real physical world, and I think that’s the vision of our entire team. And anything that can help us realize that vision faster, we’re happy to make it happen.”

Shen said the company focuses on processing its own data, rather than collecting video from a wide range of devices and sources. Once the data is obtained from the partners, Memories.ai processes and annotates the footage itself.

The company said last month that it used NVIDIA Cosmos to interpret video fed into its data lake. It then curates that data to post-train Cosmos for customers’ robotics and industrial applications. This cycle transforms real-world videos into models tailored to the tasks our customers need to solve.

The pipeline has a modular design

Memories.ai said its setup is modular to support different camera tasks and views. Shen said the company wants to grow with the industry as AI models mature for further robotics applications.

“We want to be long-term partners,” he said. “We don’t want to just sell you data. In the future, we want to work with you on models. We want to work with you on infrastructure because we know that you also are and will be limited by your resources.”

For robotics customers, Memory AI plans to tailor data to their needs, offer exclusive or non-exclusive deals, and potentially expand its role into video modeling and infrastructure.