Sony's New AI Patent Predicts Button Presses via Finger-Camera Tech

Sony's latest patent, WO2025010132, titled "TIMED INPUT/ACTION RELEASE," aims to revolutionize gaming by reducing latency in future hardware. This innovative approach, first spotted by Tech4gamers, leverages an AI model and additional sensors to predict and streamline user inputs, enhancing the responsiveness of games.

With the introduction of PlayStation Spectral Super Resolution (PSSR) in the PlayStation 5 Pro, Sony has shown its commitment to enhancing visual quality through upscaling to 4K. However, newer graphics technologies like frame generation can introduce additional latency, which can affect the responsiveness of games. This trade-off between higher frame rates and increased latency is a challenge that GPU manufacturers like AMD and Nvidia have attempted to address with solutions like Radeon Anti-Lag and Nvidia Reflex. Sony's new patent suggests that the company is exploring its own method to tackle this issue.

This new Sony patent could be a game-changer for PlayStation. Image credit: Sony Interactive Entertainment.

Sony's patent filing explains the need for this technology: "there can be latency between the user's input action and the system's subsequent processing and execution of the command. This in turn results in delayed execution of the command and unintended consequences in the game itself." The proposed solution involves a machine-learning AI model that predicts the next input a player will make, combined with an external sensor, such as a camera focused on the controller, to anticipate user commands. The patent suggests, "In one particular example, the method may include providing camera input as an input to a machine learning (ML) model. The camera input may indicate the first user command."

Additionally, Sony explores the possibility of using controller buttons as sensors, potentially integrating analog buttons in future controllers to further refine input prediction. While the exact implementation of this technology in the PlayStation 6 is uncertain, as patent filings often are, it underscores Sony's dedication to minimizing latency without sacrificing the responsiveness that gamers demand, particularly in the context of popular rendering technologies like FSR 3 and DLSS 3, which can add frame latency.

This advancement could significantly enhance gameplay in genres like twitch shooters, where both high frame rates and low latency are crucial. Whether this patent will translate into tangible improvements in future Sony hardware remains to be seen, but it certainly signals a promising direction for reducing latency in gaming.