Sony Patents In-Game Sign Language Translator

Sony has taken a significant step towards enhancing gaming accessibility for deaf players by filing a patent for an in-game sign language translator. This innovative technology aims to bridge communication gaps by translating different sign languages within video games.

Sony Patents ASL to JSL Translator for Video Games

Sony's patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," introduces a system designed to facilitate real-time sign language translation during in-game interactions. The technology focuses on converting American Sign Language (ASL) into Japanese Sign Language (JSL), enabling seamless communication between deaf gamers who use different sign languages.

Sony's proposed system works by capturing the sign gestures of one language, converting them into text, then translating that text into another specified language, and finally rendering the translated data into the corresponding sign gestures of the target language. This process ensures that players can communicate effectively regardless of their native sign language.

"Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English)," Sony explained in the patent. "Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

To implement this technology, Sony suggests using VR devices or head-mounted displays (HMDs). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," Sony detailed. "In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

Furthermore, Sony proposes that user devices can communicate over a network with a game server, which manages a shared session of the video game. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony stated, "and to which the user devices are synchronized regarding the state of the virtual environment."

This setup allows users to interact within the same virtual environment over a shared network or server. Sony also mentioned that the game server could be integrated into a cloud gaming system, which "renders and streams video" between each user device, enhancing the overall gaming experience.

By introducing this sign language translation system, Sony aims to create a more inclusive gaming environment, ensuring that deaf gamers can fully participate and enjoy their gaming experiences without language barriers.