How we recovered from the error 2070 “File check failed: File state internal error for ‘Primary’ database partition”
A few days ago while doing our regular system checks to our hosted Exchange environment we noticed that one of our Exchange 2016 servers had switched to the passive databases.
Further investigation showed that the server had been hit by the “blue screen of death” and self re-started damaging in this case the index of one of the databases running in the server. The full error message is:
Error 2070 – The Microsoft Exchange Replication service encountered an error while inspecting the logs and database for <db name>\<server name> on startup. Error: File check failed : File state internal error for ‘Primary’ database partition: m_latestIncrementalBackupTime is set, but m_latestFullBackupTime is not set
Running the command Get-MailboxDatabaseCopyStatus in the server with problems showed the server status to be ServerDown for all the databases a confirmation of the problem and in this case it had switched over to the passive.
We use our own cloud hosted backup services with Asigra infrastructure to run continuous backups of both the servers’ file system and each of the Exchange DAG databases so in this case the decision was whether to restore the damaged database from our backup system or get a copy of the passive databases and reseed it to bring it back online.
I looked at the databases running on the DAG passive server and they were active and no issues with the mailboxes were being reported therefore I decided that the correct step in this situation was to copy the passive database, reseed it and bring it back online. That’s the whole point of setting up a DAG and replicate the databases to multiple servers anyway.
With the decision made these are the steps taken in case anyone else runs into a similar situation.
Recovering the Exchange database from a passive copy
In order to copy the database I issued the following commands in the Exchange Management Shell:
- Suspend-MailboxDatabaseCopy –Identity <db name>\<primary server> –SuspendComment “Maintenance on <server name>” -Confirm:$False
- Update-MailboxDatabaseCopy –Identity <db name>\<primary server> –SourceServer <remote server> –DeleteExistingFiles
- Resume-MailboxDatabaseCopy –Identity <db name>\<primary server>
I ran the command Get-MailboxDatabaseCopyStatus and now the databases were showing a Healthy status but the Content Index State for the one database was showing as failed.
I tried to re-seed the catalog only running Update-MailboxDatabaseCopy -Identity \ -SourceServer -CatalogOnly -BeginSeed but the Content Index State was still showing as failed but no errors were showing in the Windows event log.
I went to the Exchange admin center, selected the problem database and selected Activate. Ran Get-MalboxDatabaseCopyStatus in the Exchange Management Shell and now all databases were showing Healthy and no errors in the Content Index State.
Hope this information helps someone else out there. If you have any questions please leave a comment or contact us at firstname.lastname@example.org.