The cornerstone of the PKI is the private key you use to encrypt or digitally sign information. One of the most significant things a PKI has to offer is non-repudiation. Non-repudiation guarantees that the parties involved in a transaction or communication cannot later on deny their participation. Imagine how vital this requirement is for e-commerce; as a consumer or a seller you have warranties that you will get what you have paid for or even that you will be paid as agreed.  

Assuming that your private key is stolen, this will lead to the compromise of the authentication and non-repudiation gained through PKI because the attacker can use your private key to digitally sign documents and pretend to be you. This is obviously a situation you do not want to face. Imagine how much damage (economically, credibility etc.) can be caused by someone else running around masquerading as you!    

Compromising the private key is a threat that involves not only the holders of the digital certificate but the CA itself. Compromising the CA’s private key may lead to dramatic consequences if it is not detected immediately. The attacker can use the CA’s private key to generate numerous fraudulent digital certificates that may then be used for illegal purposes. 

How to minimize the risk: Both the holders of a digital certificate and the issuing authority must protect the computers, storage devices or whatever they use to keep the private keys. A combination of security solutions can be used to achieve a high-level of protection such as strong passwords, anti-virus, firewalls, intrusion detection tools etc.