A context-aware distributed firewall scheme is provided. A firewall engine tasked to provide firewall protection for a set of network addresses applies a reduced set of firewall rules that are relevant to the set of addresses associated with the machine. A hypervisor implements a search structure that allows each virtual machine's filter to quickly identify relevant rules from all of the received rules. The search structure is constructed as a binary prefix tree, each node corresponding to an IP CIDR (Classless Inter-Domain Routing) block. A query for relevant rules traverses nodes of the search structure according to a queried IP address and collect all rules that are associated with the traversed nodes.