Suppose that I have some application servers writing to two databases using two-phase-commits to keep the data consistent. As long as both databases are up everything is fine.
But now one database goes down. The application servers recognize it and continue by writing just to remaining database. How to get the dead database up again?
Two things have to happen: first the broken database must be synchronized with the running one and second exactly after the databases have been synchronized the application servers need to write again to both databases. How it this possible without stopping the application servers?