A learning protocol for distributed antenna state selection in directional cognitive small-cell networks is described. Antenna state selection is formulated as a nonstationary multi-armed bandit problem and an effective solution is provided based on the adaptive pursuit method from reinforcement learning. A cognitive small cell testbed, called WARP-TDMAC, provides a useful software-defined radio package to explore the usefulness of compact, electronically reconfigurable antennas in dense small-cell configurations. A practical implementation of the adaptive pursuit method provides a robust distributed antenna state selection protocol for cognitive small-cell networks. Test results confirm that directionality provides significant advantages over omnidirectional transmission which suffers high throughput reduction and complete link outages at above-average jamming or cross-link interference power.