System and methods are described to create successive abstractions of real and virtual environments and objects. A common framework is provided to define abstractions for a large collection of various input data feeds. It is assumed that environments are either instrumented to produce such messages or that they may contain sensors or smart devices that generate such messages. Incoming messages are analyzed preserving latency requirements and without imposing unduly heavy procedures. The analysis yields various abstractions at several scales that can be utilized by application programs. A system architecture using specialized storage mechanisms is proposed that preserves the latency requirements of the incoming data messages and the generated abstractions.