Sandbox security is complementary to honeypots. It usually involves a special VM that is kept in isolation from the rest of your network resources. Its sole purpose is to be a miniature laboratory to observe malware behavior. Security researchers have been using such sandboxes to analyze malware for many years. Because the sandbox is a controlled environment, its code can be dissected line by line without worrying about potential harm to other computers.
You can read my post on Network Solutions’ blog here, where I talk about its evolution and some of the online sources of sandboxes that can be used for testing for the presence of malware. Sandboxes also play a key role in the escalating war of obfuscation and detection evasion by malware.