HomeOracle Page 9 - Oracle and Availability: Illustrated Downtime Scenarios
Complete and Total Disaster - Oracle
Ever have to recover your servers from a flooded basement? Move a database to an new platform and need the weekend to test it but users need the database today? We will illustrate various database problems and which piece of technology could be employed to prevent the outage or to recover from it quickly. (From the book Oracle Database 10g High Availability with RAC, Flashback & Data Guard by Matthew Hart and Scott Jesse, ISBN: 0072254289, McGraw-Hill/Osborne, 2004.)
Disaster struck Horatio's Woodscrews on a particularly wet spring day. Rain had been falling steadily for weeks in the city. The DBA was sleeping soundly, dreaming of data protection systems, when his pager began to beep. He called the number groggily, looking at the clock. It was the middle of the night.
A water main had busted near the Woodscrew building, and the basement had flooded. The basement held all of the database systems—the production Solaris box, the Sales RAC cluster, the test and dev servers, everything. The utility company was marking the area as a disaster area, and not letting anyone near the building.
All the servers were lost. The DBA had to think fast. Did they move the tape backups off-site, as was proposed a few months back? He could not remember, so he called the backup administrator. Did they have those old Solaris boxes they'd retired last year someplace? The DBA dialed the system administrator.
A conference call was put together in the middle of the night. Where were the archived tape backups? Were they moved to a different building? When was the last time the tapes were moved to the other location? How much data would be lost in the flood?
The questions flew around to the system administrator—Where could they find new systems to install Oracle? Would there be enough disk space without the SAN device? Should they contact the vendors to get emergency hardware shipped in?
Then the questions came to the DBA: Could we rebuild the data once the drives were salvaged from the flood? Could we hobble along on one of the older servers at the other office? How fast could he get Oracle installed, patched, and ready to start the file restore from the archive tapes?
And, finally, the ultimate question: Who's going to call the CEO and tell him what happened?
Where to Go from Here
The downtime scenarios that we have tried to illustrate in this chapter are but the tip of the iceberg. We wanted to show you that there are common, everyday situations that can be fixed using new functionality in Oracle Database 10g. Often, this is functionality that you already have at your fingertips and can leverage right now, with just a few steps that we outline in the following chapters.
We also wanted to show you some of the more disruptive events that can immobilize a database: user errors, such as a truncated table, or a hardware failure. These types of events happen less frequently, but the outage can be extremely long without careful planning that accounts for the possibility. From RAC to Flashback, careful planning and forethought will guide you through these disruptions.
Protect Yourself Against Disaster with Data Guard
Oracle Data Guard offers the most effective protection against complete site failure, with real-time data push from the primary database to the standby database. Data Guard allows you to activate an exact copy of the primary database in just a few minutes (or seconds), so that the outage is almost unnoticeable. It can be configured in many different ways to meet the business demands you may have. The standby database can be used as a reporting database, for taking backups of the production database, or even as a failover site during massive system reconfiguration at the primary site. For a complete rundown of the flexibility and usability of Oracle Data Guard, see Chapter 7.
Even less predictable, and less frequent, is the complete loss of a system due to natural disaster or site network outage. In situations where the entire database is lost and cannot be quickly restored, there are important questions that begin to circulate, and some of them have to do with who's at fault for the lack of a disaster plan. Oracle Data Guard offers one of the most advanced disaster-proofing solutions on the market.
So, now that you've explored just a few of the situations that might cause an outage, its time to explore the technologies that can prevent (or significantly reduce) those disruptive downtimes.
This chapter is from Oracle Database 10g High Availability with RAC, Flashback & Data Guard, by Hart and Jesse. (McGraw-Hill/Osborne, 2004, ISBN: 0072254289). Check it out at your favorite bookstore today. Buy this book now.