User interface virtualization describes a technique for providing a user with access to one computing device from another computing device, while translating the ergonomics of one computer's user interface style into the ergonomics of the other's. An agent running on a remote desktop collaborates with a corresponding client running on a client machine that accepts a “touch and swipe” style input. The agent and client exchange user interface metadata and user interface input events and translate the exchanged information to provide native graphical user interface elements (at the client machine) and simulated user actions (at the remote desktop). The agent running on the remote desktop may use an interface interaction API or library to programmatically manipulate the user interface of the remote desktop responsive and act as a proxy for the corresponding client.