Research
I work on world models with machine learning and deep learning, focusing on prediction in the real world and on the representations that support it.
Prediction and Verbalization of Physical Environments
Predicting what happens next in real-world scenes, and describing the prediction in natural language.
When humans understand the real world, we capture change points of the objects we observe (a collision, a bounce) and understand the result as a linguistic expression. Following this idea, I built prediction models that handle physical information such as object positions and velocities alongside the images themselves, so that the moment of change can be detected. The inferences made by the model are then generated as sentences such as “a green sphere is repulsed by a red cylinder”, which lets us examine what the model actually predicted.
Related work: JACIII (2025) ・ JACIII (2025) ・ ICMLC2025 ・ SCIS&ISIS2024 ・ DS2023 ・ PRML2023
Learning Abstract Action Representations (Latent Actions)
Learning action representations from observation sequences alone, and organizing them into abstract units grounded in motion semantics.
Many world models focus primarily on predicting future observations, treating actions merely as inputs that drive state transitions; the semantic structure of actions is therefore not modeled explicitly. In this work, changes between consecutive observations are encoded into latent representations using an inverse dynamics model. Contrastive learning based on motion categories encourages semantically similar actions to be embedded close to each other across different agents, and category-wise center representations are then treated as discrete action units.
Related work: JSAI 2026
Current Interests
Building on prediction in the real world and on learning action representations, I am interested in how world models can generalize to unseen objects and situations.
I would also like to move beyond in-lab experiments toward validation in real environments and field studies.
Coming soon