You become a citizen as soon as you take the Oath of Allegiance to the United States. In some places, you can choose to take the Oath the same day as your interview.
You may get a new Certificate of Naturalization by submitting an "Application for Replacement Naturalization/Citizenship Document" (Form N-565) to USCIS.