Any record that you maintain can theoretically be obtained by others, either through burglary, by subpeona, or with a search warrant. The best way to be sure your records wont come into the wrong hands is to actually destroy them. If something doesn’t exist it can’t be stolen!
There are certain legal requirements to store some types of records. For example, the IRS has recommendations for tax record retention. In addition, if you destroy records knowing that the government is trying to see them, you can be charged with Tampering with Evidence. However, it is not a crime if you destroy the same records before you are aware of any investigation.
Make a policy of routinely destroying any paperwork or records which you do not have a specific need to maintain. This policy should apply to both paper and electronic documents.