Showing posts with label Recovery. Show all posts
Showing posts with label Recovery. Show all posts

2021-05-13

Recovery Point Objective with a time perspective

 Usually the Recovery Point Objective (RPO) is only discussed as the data loss in the time between the last backup and the incident. But on most systems a point-in-time restore is not relevant after some time. After some time and backup generations most backups are becoming irrelevant, and can be deleted to save storage. A few backup are then kept for audit purpose and long term investigations.

A RPO plan could look lige this:

  1. 0 to 42 days: Point-in-time
  2. 42 days to 3 years: Week
  3. 3 years to 10 years: Month
  4. 10 years to 20 years: Quarter
Cleaning up backup data is also called remediation  of backup data.

Cleaning up data is not only the backup sets but also the metadata on the backups. But the metadata on the backup duration, size of backup set etc. could be useful with a high degree of detail for a longer time that the backup sets.

2012-10-16

Check NetBackup file backup


I have been looking in ways to ensure that a SQL Server backup file is backed up by NetBackup, so that the backup file can be deleted from the local disk. I have not found a API for the local NetBackup client, but there are a lot of command-line tools.
In this case our backup vendor has pointed out the tool "bplist" (bplist.exe). Usually bplist is located in the folder „%ProgramFiles\Veritea\NetBackup\bin\“.
A bplist answer example looks like this:
-rwx------ root;SQL1 root;Doma      485888 Jul 29 20:00:01 E:\MSSQL\Backup\ReportServer_backup_2012_07_28_221142_2861771.bak
if bplist is called with the parameters „-l -b -Listseconds“.

WARNING!!!
No year on NetBackup backup.
This is also discussed in the forum thread "bplist does not display the year of a backup?".
It looks like there is an issue on the age of the backup, if it is more than six months old or not.
Also when I look in the documentation, there are some differences on the answer from the NetBackup server if it is Windows or Linux. Please notice that it is the NetBackup server, not the client operating system that gives the answer.

bplist can be executed in PowerShell with the invoke operator (&):
& "<NetBackup folder>\bplist.exe" -l -b "X:\MSSQL\Backup\sqladmin_repository_backup_2012_12_24_172249_7006410.bak"
The full path must be provided for the file to examine in NetBackup.

To look at how to handle bplist and its answer in a automated way, I made this PowerShell spike script:
param (
  [Parameter()]
  [ValidateScript({Get-ChildItem -Path $_})]
  [string]$NetBackup_Folder = 'C:\Program Files\Veritas\NetBackup\bin'
)

Set-StrictMode -Version 2.0

function Get-NetBackup_bplist {
[CmdletBinding()]
param(
  [Parameter(Mandatory=$true,ValueFromPipeline=$true)]
  [System.IO.FileInfo]$File
)
  BEGIN { Write-Verbose "OS Version = $([System.Environment]::OSVersion.VersionString)" }
  PROCESS {
    Write-Verbose "File name = '$($File.FullName)'."

    $BpList = New-Object -TypeName PSObject
    $BpList = $File
    $BpList.PSObject.TypeNames.Insert(0,'SqlAdmin.NetBackup.BpList')

    # Get file backup status from NetBackup
    $bplist_answer = $(& "$NetBackup_Folder\bplist.exe" -l -b -Listseconds "$($File.FullName)") 2>&1 # Redirect bplist.exe error to $bplist_answer
    if ($bplist_answer.GetType().IsArray) { # Multiple backups in NetBackup
      Write-Verbose " $($bplist_answer.Length) backups found in NetBackup. Will continue on last backup."
      $_bplist = $bplist_answer[0] # Get last backup in NetBackup
    }
    else {
      $_bplist = $bplist_answer
    }
    Write-Verbose " $($_bplist)"

    # Evaluate file backup status from NetBackup. Add -PassThru to last added member.
    if ($_bplist.ToString() -ceq 'EXIT STATUS 227: no entity was found') { # File not in NetBackup
      Write-Verbose " --- NO backup in NetBackup ($($File.Name))."
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name HasBackup -Value $false
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name BackupLength -Value $null
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name BackupTime -Value $null
    }
    else {
      Write-Verbose " +++ Backup is in NetBackup ($($File.Name))."
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name HasBackup -Value $true

      # Get backup details
      $regex = [regex]"\w+"
      $Backup = $($regex.matches($_bplist)) # Returns [System.Text.RegularExpressions.Match]

      if ($Backup[5].Value.SubString($Backup[5].Length-1) -eq 'K') { # Is the most right char 'K'?
        $Backup_Size = [int]$Backup[5].Value.SubString(0,$Backup[5].Length-1) * 1024 # Convert from KB to Bytes
      }
      else {
        $Backup_Size = [int]$Backup[5].Value
      }
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name BackupLength -Value $Backup_Size
      $DateParse = "$($Backup[7].Value) $($Backup[6].Value) $([System.DateTime]::Now.Year) $($Backup[8].Value):$($Backup[9].Value):$($Backup[10].Value)"
      Add-Member -InputObject $BpList -MemberType NoteProperty -Name BackupTime -Value $([System.DateTime]::Parse($DateParse))
    }
    Write-Output $BpList
  }
  END {}
}


### INVOCATION ###
switch -casesensitive ($ComputerName) {
  'TRACY.SQLADMIN.LAN' {
  Get-ChildItem -Path 'D:\MSSQL_Backup' |
  Sort-Object -Property Length -Descending |
  Sort-Object -Property LastWriteTime -Descending |
  Select-Object -First 100 |
  Get-NetBackup_bplist | #-Verbose |
  Where-Object { $_.HasBackup -eq $true } |
  #Format-Table Name,Length,HasBackup,BackupTime,BackupLength -AutoSize
  ForEach-Object { Remove-Item $_.FullName -Confirm:$true }
  break
  }
  default {
  Get-ChildItem -Path 'X:\MSSQL\Backup' |
  Sort-Object -Property Length -Descending |
  Select-Object -First 20 |
  Get-NetBackup_bplist |
  #Format-Table Name,Length,HasBackup,BackupTime -AutoSize
  ForEach-Object { Remove-Item $_.FullName -Confirm:$true }
  break
  }
}


Again – a spike…

The general motivation to look into this is to ensure that the restore chain is complete, also in the secondary backup on NetBackup.
The complete restore chain is necessary to ensure complete recovery.

(This is a running update on a post from 2012-08-27)

2012-04-09

Get Backup Directory with ADO.NET

About 1½ year ago I made a entry on this blog on how to get the SQL Server Backup Directory by using the undocumented stored procedure "[master].[sys].[xp_instance_regread]".
Today I needed the path name in a PowerShell script, and I also wanted to call the procedure correct.
This I have done by calling the procedure through ADO.NET as a stored procedure, not in a EXECUTE statement as dynamic SQL.
$ServerName = '(local)'
$cnnStr = "Data Source=$ServerName;Integrated Security=SSPI;Application Name=SqlBackupFolder"
$Cnn = New-Object System.Data.SqlClient.SqlConnection $cnnStr
$Cmd = New-Object System.Data.SqlClient.SqlCommand
$Cmd.Connection = $Cnn
$Cmd.CommandText = '[master].[sys].[xp_instance_regread]'
$Cmd.CommandType = [System.Data.CommandType]::StoredProcedure
$Cmd.Parameters.Add("@rootkey", [System.Data.SqlDbType]::NVarChar, 128) | Out-Null
$Cmd.Parameters['@rootkey'].Direction = [System.Data.ParameterDirection]::Input
$Cmd.Parameters['@rootkey'].Value = 'HKEY_LOCAL_MACHINE'
$Cmd.Parameters.Add("@key", [System.Data.SqlDbType]::NVarChar, 128) | Out-Null
$Cmd.Parameters['@key'].Direction = [System.Data.ParameterDirection]::Input
$Cmd.Parameters['@key'].Value = 'SOFTWARE\Microsoft\MSSQLSERVER\MSSQLSERVER'
$Cmd.Parameters.Add("@value_name", [System.Data.SqlDbType]::NVarChar, 128) | Out-Null
$Cmd.Parameters['@value_name'].Direction = [System.Data.ParameterDirection]::Input
$Cmd.Parameters['@value_name'].Value = 'BackupDirectory'
$Cmd.Parameters.Add("@value", [System.Data.SqlDbType]::NVarChar, 128) | Out-Null
$Cmd.Parameters['@value'].Direction = [System.Data.ParameterDirection]::Output
$Cnn.Open()
$_RowCount = $Cmd.ExecuteNonQuery()
$Cnn.Close()
$SqlBackupFolder = $Cmd.Parameters['@value'].Value
":: SQL Server Backup Folder = '$SqlBackupFolder'."

The answer from the script is like
:: SQL Server Backup Folder = 'C:\MSSQL\Backup'.

It is possible to reduce the number of lines in the script, but this way I can use the call of the procedure for other information than the Backup Directory.

If you plan to reuse the Command object, you should consider to remove the Parameters ($Cmd.Parameters.Clear()) and reset the CommandType ($Cmd.CommandType = [System.Data.CommandType]::Text).

2010-08-23

Backup Directory

With SQL Server 2005 we were able to define a default backup directory. This is a nice thing to maintain, so that a restore can be done quickly and correct.
The directory path is stored in the Windows Registry, and is available with a (undocumented) stored procedure.
DECLARE @reg_key_data NVARCHAR(255);
EXECUTE [master].[dbo].[xp_instance_regread]
  N'HKEY_LOCAL_MACHINE'
 ,N'SOFTWARE\Microsoft\Microsoft SQL Server\MSSQLServer'
 ,N'BackupDirectory'
 ,@reg_key_data OUTPUT;
SELECT @reg_key_data AS N'backup_directory';

(Fig. 1) SQL Server Management Studio, Facets

The script works on SQL server 2005 and 2008.
You could set the backup directory path by the (undocumented) stored procedure master.dbo.xp_instance_regwrite, but I prefer to change the value at the installation or manually using SQL Server Management Studio.
The name of the backup directory is available for read and write in SQL Server Management Studio as a facet to the database instance. Right-click on the instance in Object Explorer and select Facets. In the facet "Server" it is the second item.

Reference

Dinesh Asanka: „Working with the Registry“ (SQLServerCentral).

2010-08-18

Set Recovery Model Simple

On a test installation with quite a few databases we decided to alter the Recovery Model to Simple.
This statement generated the ALTER statements.
SELECT N'ALTER DATABASE [' + [name] + '] SET RECOVERY SIMPLE WITH NO_WAIT;' AS N'DDL'
FROM [master].[sys].[databases]
WHERE [recovery_model] <> 3 AND [name] <> N'model';

The output is copied to the (same) query window in SQL Server Management Studio and executed.
The total time of delivery is measured in seconds :-)

2008-09-15

Restore Workshop draft

Some day - no, week(-end) - I would like to gather some comrades in arms - SQL Server naturally! - and do a workshop on SQL Server database restore.
The initial list of subjects is like this:
  • Restore on full backup
  • Restore on log backup
  • Restore on differential backup
  • Restore point-in-time
  • Restore online
  • Restore Full-Text index
  • Restore Filestream
  • All above on both single and multiple filegroups
  • Restore msdb
  • Restore master
  • Rebuild database instance
  • Rebuild Windows Server
  • Discuss how to get, store and access the actual version of operating system, applications, .NET, MDAC etc.
  • Switch center using clustering, database mirroring and log shipping
  • Validate backup - discuss how to do this automatic
  • Discuss device file versus backup files
  • Recover using snapshot
  • Discuss documentation like how, where, what, tools, access and maintenance
  • Discuss security about backup sets and snapshots and access to these
  • A general discussion about SQL Server infrastructure from the viewpoint of recovery and restore
I would like the workshop to be held every year. The participants should be grouped in teams of two or three persons, not more or less.
Each group should have one workstation and three servers - virtual servers could do, but there should be enough disk capacity to create several partitions for each server.
The workshop should be placed away from a major city, so that the participants are not tempted to drop in and out... Of course the food and the housing should support the process and the good spirit.
The seats should be distributed to persons that did not participate before those did the subjects last year.

I think it could be fun and most enlightening!

Paul Randal has made list of good questions to ask yourself - „SQLskills SQL101: Practicing disaster recovery

History

2008-09-15 Post created on old notes.
2017-07-18 Reference to Paul Randal added.