Over the years I’ve discussed log space reservation, which is when SQL Server automatically reserves some free space in the transaction log so that in-flight transactions can always be rolled back without the log having to grow. This is because rolling back a transaction requires generating more log records, and so there needs to be guaranteed space for them. If this did not happen, the log could fill up, in-flight transactions would begin to roll back, a log grow attempt might fail, and the database then goes suspect or into recovery because the in-flight transactions are essentially stuck.
The algorithm is pretty conservative, and I fixed a few bugs in it before SQL Server 2005 shipped.
There hasn’t been a case of it failing to reserve enough space until SQL Server 2012, when a bug was introduced. That bug was discovered by someone I was working with in 2015 (which shows just how rare the circumstances are), and at the time it was thought that the bug was confined to the log of tempdb filling up, rollback failing, and the server shutting down.
However, just last week I was contacted by someone running SQL Server 2012 SP3 who’d seen similar symptoms but for a user database this time, and the user database went into recovery. An example of the error log messages is below (altered for anonymity):
During under of a logged operation in database 'mydb', an error occurred at log record ID (2445:89001:23). Typically, the specific failure is logged previously as an error in the Windows Event Log service. Restore the database or file from a backup, or repair the database. The log for database 'mydb' is not available. Check the event log for related error messages. Resolve any errors and restart the database. Error during rollback, shutting down database (location: 1) Database mydb was shutdown due to error 3314 in routine 'XdesRMReadWrite::RollbackToLsn'. Restart for non-snapshot databases will be attempted after all connections to the database are aborted. The transaction log for database 'mydb' is full due to 'ACTIVE_TRANSACTION'. The transaction log for database 'mydb' is full due to 'ACTIVE_TRANSACTION'. The transaction log for database 'mydb' is full due to 'ACTIVE_TRANSACTION'. 'D:\Logs\mydb.ldf: Operating system error 112(There is not enough space on the disk.) encountered.
I suggested that they’d hit the known bug, and they confirmed that with Microsoft.
And if you hit it for tempdb, the server will shut down, as tempdb being unavailable means SQL Server has no choice but to stop.
The bug is described in KB article 2963384 and is included in SQL Server 2012 SP4 and SQL Server 2014 SP1. If you’re running 2012 SP3 then you should install SP4, and if you’re running 2014 RTM then you should install the latest 2014 SP.
I didn’t blog about the bug back in 2015 as only one person had hit it and the circumstances seemed incredibly rare, but now that seems to not be the case.
Stay safe out there!
The post 2012/2014 bug that can cause database or server to go offline appeared first on Paul S. Randal.