Guns stolen from law-abiding households provide an important source of guns for criminals. Crime rates increase with the availability of stolen guns, so the gun market is subject to externalities that generate excessive ownership and inadequate spending on protective measures to deter gun theft. We show that a simple refundable deposit for guns can internalize the externalities in the gun market and decrease gun theft and crime rates. We also show that the self-defense motive for gun ownership can lead to multiple equilibria.