The present teaching relates to method, system, medium, and implementations for user machine dialogue. An instruction is received by an agent device for rendering a communication directed to a user involved in a dialogue in a dialogue scene and is used to render the communication. A first representation of a mindset of the agent is updated accordingly after the rendering. Input data are received in one or more media types capturing a response from the user and information surrounding the dialogue scene and a second representation of a mindset of the user is updated based on the response from the user and the information surrounding of the dialogue scene. A next communication to the user is then determined based on the first representation of the mindset of the agent and the second representation of the mindset of the user.