When driving through the city, you will encounter many police officers. You can run one of them down and pick up his standard issued gun. Your wanted level will go up, but immediately diminish. This works better if there is only one police officer, on foot, with no police officer witnesses.
During free ride mode, taxi drivers and occasionally regular motorists will carry the small brown revolver and the small silver revolver. Pull them out of their cars and kill them to get the weapon.