Is it Safe To jailbreak your iPhone?

Hi! I want help with this. I am going to be getting an iPhone soon and was wondering about jailbreaking it. I have an iPod Touch 32gb 3rd gen that is jailbroken on version 4.3.1. It is running smoothly, and i have no problems with it. My actual question is, when you jailbreak your phone, and something goes wrong, and you have to restore it, will you lose all your numbers and things like that, which is stored on the microCym card?I mean, I do use backups, but i never synk my emails and other things such as contacts to my computer ever. If anyone ever had to restore their iPhone, please give me feedback on if the contacts and things like that stay. Thanks in advance. Deng