Showing posts with label SQL Server. Show all posts
Showing posts with label SQL Server. Show all posts

2025-01-05

The obscure star-database

To play around with some obscrure naming possibilities in SQL Server I create the star-database. The name has nothing to do with the data warehouse star-schema pattern. The name is because the database is named with the star character (*).

CREATE DATABASE [*]
    ON PRIMARY ( NAME = N'star_data', FILENAME = N'L:\SSDB2019\Data\star_data.mdf', SIZE = 8MB, FILEGROWTH = 8MB )
    LOG ON ( NAME = N'star_log', FILENAME = N'L:\SSDB2019\TransLog\star_log.ldf', SIZE = 8MB, FILEGROWTH = 8MB );

The database is created on specific directory as inplicit default path using only the database name does not work. This was expected. Using the star-character as name requires me to wrap the name in [].

In the star-database I then create a star-schema.

USE [*];
GO
CREATE SCHEMA [*] AUTHORIZATION [dbo];

... and then a star-table.

CREATE TABLE [*].[*]
(
    [*] bigint NOT NULL IDENTITY (0, 2),
    [%] nvarchar(42) NULL,
    [<] int null,
    [>] int null
);
GO
ALTER TABLE [*].[*] ADD CONSTRAINT
    [PK_star] PRIMARY KEY CLUSTERED 
    ([*]) ON [PRIMARY];

The column names are are almost as obscure as the other names. And then I insert a few rows.

INSERT INTO [*].[*] ([%],[<],[>])
VALUES
    (N'række0', 1000, -1000),
    (N'rÆkke1', 10, -100),
    (N'räkke2', 237, 7);

Then I can select the table in different ways, where the syntax is quite important to the output.

SELECT [*] FROM [*].[*];
SELECT * FROM [*].[*];
SELECT COUNT(*) AS [**] FROM [*].[*];

The first two statements visually looks similar, but the [] makes quite a difference. Adding some WHERE-clauses also adds to the fun.

SELECT * FROM [*].[*] WHERE [%] LIKE '%ä%';
SELECT * FROM [*].[*] WHERE [<]<500;
SELECT * FROM [*].[*] WHERE [>]<0;

The script is in SQLAdmin Github (star_database.sql). Next round could be adding more fun with unicode...

The obscure master-database

The master database is well known. And creating a database with a similar name can make things obscure or unexpected. Playing with adding space before or after "master" is only partly possible. Adding a space after the name as [master ] does not work and gives the error "Msg 1801, Level 16, State 3, Line 8 Database 'master ' already exists. Choose a different database name.". But adding a space before like [ master] does work.

CREATE DATABASE [ master] ON  PRIMARY 
( NAME = N'master0_data', FILENAME = N'L:\SSDB2019\Data\master0_data.mdf', SIZE = 8MB, FILEGROWTH = 8MB )
LOG ON 
( NAME = N'master0_log', FILENAME = N'L:\SSDB2019\TransLog\master0_log.ldf', SIZE = 8MB, FILEGROWTH = 8MB );

Adding other characters known from other parts of a database can add to the obscure situation.

CREATE DATABASE [master.] ON  PRIMARY 
( NAME = N'master_dot_data', FILENAME = N'L:\SSDB2019\Data\master_dot_data.mdf', SIZE = 8MB, FILEGROWTH = 8MB )
 LOG ON 
( NAME = N'master_dot_log', FILENAME = N'L:\SSDB2019\TransLog\master_dot_log.ldf', SIZE = 8MB, FILEGROWTH = 8MB );
GO

CREATE DATABASE [master.dbo] ON  PRIMARY 
( NAME = N'master_dbo_data', FILENAME = N'L:\SSDB2019\Data\master_dbo_data.mdf', SIZE = 8MB, FILEGROWTH = 8MB )
 LOG ON 
( NAME = N'master_dbo_log', FILENAME = N'L:\SSDB2019\TransLog\master_dbo_log.ldf', SIZE = 8MB, FILEGROWTH = 8MB );

The script file with the obscure master databases is also in SQLAdmin Github in obscure_master_database.sql.

2024-11-19

List orphan database users

Some days ago I was asked - politely - to move some databases between two SQL Server installations. Unfortunately the service had failed on the source, so I have no access to the logins definitions.
This give me a lot of orphan database users.
To make thing more entertaining some of the databases has compability level 80, and this can not be changed.
To get a list of orphan database users, I have to combine two methods. This is because the compability level 80 databases does not have the system view database_principals.

DECLARE @db_name AS nvarchar(128); -- Database name
DECLARE @_msg AS nvarchar(2047); -- Execution message
DECLARE @_cmd AS nvarchar(MAX); -- Dynamic command
CREATE TABLE ##orphan_user (
[db_name] nvarchar(128) NOT NULL,
[user_name] nvarchar(128) NOT NULL
);

DECLARE db_cursor CURSOR FAST_FORWARD FOR
SELECT [name]
FROM [master].[sys].[databases]
WHERE [database_id] > 4;

OPEN db_cursor;
FETCH NEXT FROM db_cursor INTO @db_name;
WHILE @@FETCH_STATUS = 0
BEGIN
IF (SELECT [compatibility_level] FROM [master].[sys].[databases] WHERE [name] = @db_name) < 90 BEGIN SET @_msg = N'The database [' + @db_name + N'] is SQL Server 2000 or older.'; SET @_cmd = 'INSERT INTO ##orphan_user ' + N'SELECT ''' + @db_name + N''' AS [db_name], [sysusers].[name] AS [user_name] ' + N'FROM [' + @db_name + N'].[dbo].[sysusers] ' + N'LEFT OUTER JOIN [master].[dbo].[syslogins] ON [sysusers].[sid] = [master].[dbo].[syslogins].[sid] ' + N'WHERE [sysusers].[uid] > 4 AND [sysusers].[issqlrole]=0 AND [sysusers].[isapprole]=0 AND [master].[dbo].[syslogins].[sid] IS NULL;';
END
ELSE
BEGIN
SET @_msg = N'The database [' + @db_name + N'] is SQL Server 2005 or newer.';
SET @_cmd = N'INSERT INTO ##orphan_user ' +
N'SELECT ''' + @db_name + N''' AS [db_name], [database_principals].[name] AS [user_name] ' +
N'FROM [' + @db_name + N'].[sys].[database_principals] ' +
N'WHERE [database_principals].[type] IN (''S'',''U'',''G'') ' +
N'AND [database_principals].[is_fixed_role] = 0 ' +
N'AND [database_principals].[name] NOT IN (N''dbo'', N''guest'', N''sys'', N''INFORMATION_SCHEMA'');';
END
RAISERROR(@_msg,0,0) WITH NOWAIT;
EXECUTE (@_cmd);

FETCH NEXT FROM db_cursor INTO @db_name;
END

CLOSE db_cursor;
DEALLOCATE db_cursor;


SELECT * FROM [##orphan_user];

SELECT DISTINCT [user_name] FROM [##orphan_user]
ORDER BY [user_name] DESC

DROP TABLE [##orphan_user];


I execute only a part of the script where I fill the temporary table. Then I can filter and sort the data to handle the logins.

2024-10-28

Put SQL Server PVS in own filegroup

Putting the Persisted Version Store (PVS) in its own file group is to improve the performance of SQL Server Accelerated Database Recovery (ADR). ADR was introduced with SQL Server 2019 and had some improvements with SQL Server 2022. This is described in the Microsoft documentation "Accelerated database recovery". Putting the PVS in its own filegroup is recommended and described in the Microsoft document "Manage accelerated database recovery".

This example is on the AdventureWorks database. The script i in the SQLAdmin Github repository.

Please notice that the Microsoft documentation does not create the filegroup before using it for PVS. This is required.

2024-09-09

SQL Server Workstation

Some notes from a SQL Server 2022 Developer Edition installation on a Windows 11 Pro workstation. The infrastructure is VMware Workstation Pro.

General Preperations

If you clone an exiting VM then you should rename the new SQL Server workstation to something different. And do it before anything else.

As I usually has a sandbox for different versions of SQL Server the name could something like "MSSQL 2022 Workstation". Consider if you should differ between sandbox, workstation, devbox or anything else.

Make sure the Windows installation is up to date.

PowerShell

Install PowerShell and then Windows Terminal from Microsoft Store.

Open Windows Terminal and pin it to the task bar. Then close Windows Terminal.

PS > Install-Module -Name SqlServer
PS > Install-Module -Name dbatools

Storage

Move CD-ROM to Z–drive.

DriveNameSizeNTFSPathComments
CSystem64 GBdefault Windows and tools (SSMS)
PProgram64 GBdefault SQL Server
DData64 GB64 KB User database (mdf & ndf)
LTranslog32 GB64 KB User db & tempdb (ldf)
Ttempdb32 GB64 KB tempdb (mdf & ndf)
XBackup128 GB64 KB Full, diff & log backup
  1. Create one disk at a time: One-Piece Flow (Lean)- Must be done manually in VMware Workstation. Can be scripted in vCenter with PowerCLI.
  2. In VMware Workstation NVMe can only be added offline.
  3. Split virtual disk into multiple files.
  4. Do not allocate all space now.
  5. „MSSQL Workstation_<Drive>“
  6. All partitions are GPT.
  7. Only use Simple Volume.
  8. Quick format.

SQL Server Installation

  1. Connect VM CD-drive to SQL Server ISO-file
  2. Run SETUP.EXE
  3. Select „SQL Server Developer Edition“
  4. Select "Database Engine Services" and "Full-Text…"
  5. Place "Instance Root Directory" on another physical drive than Windows, e.g. "P:\Program_Files\Microsoft_SQL_Server\". This is to spread preassure on (local) storage units.
  6. Name the (Database Engine) instance, e.g. „SSDB00“. This will allow side-by-side installation of different SQL Server versions. On the other hand I usually does not recommend instanse name with version information. The instance name in shared installations should be generic – like a dumb database key.
  7. Change Service Startup Type to "Manual". When running several instances this will help you control local resources on your workstation. With virtual workstations I usually create one VM to each instance.
  8. Set „Grant Perform Volume Maintenance Task priviledge to SQL Server Database Engine Service“. This is to speed up database file (auto-)growths which might be when expanding the physical data model. This is really relevant on a workstation where you experiment. On the other hand I would never use Auto Shrink (AUTO_SHRINK) even in a sandbox.
  9. Collation tab: Use collation "Latin_General_100 AS KS WS" maybe with UTF-8.
  10. Add Current User as SQL Server administrator. Adding (Local) Administrators is not enough.
  11. Set Directories. Define seperate paths for
    • Data
    • TransactionLog
    • Backup
    • tempdb
      where [tempdb] transaction log is with the other user database transaction logs.
  12. Set tempdb data and log
  13. Set MAXDOP to 4
  14. Set memory to Recommended
  15. Run installation
  16. Close SQL Server Installation
  17. Disconnect CD-drive
  18. Restart VM

SQL Server Update

  1. Copy latest CU to VM. I use „C:\temp\“
  2. Start PowerShell as Administrator. That is Microsoft PowerShell (v7) not Windows PowerShell (v5)
  3. Change directory to „C:\temp\“
  4. Run the SQL Server CU exe-file
  5. Complete SQL Server CU installation
  6. Delete SQL Server CU exe-file from „C:\temp\“
  7. Restart VM

SSMS Installation

  1. Copy latest SSMS installation (SSMS-Setup-ENU.exe) to „C:\temp\“
  2. Start PowerShell as Administrator
  3. Change directory to „C:\temp\“
  4. Run SSMS Setup exe-file
  5. Complete SSMS installation
  6. Delete SSMS Setup exe-file from „C:\temp\“
  7. Start SSMS
  8. Cancel suggested connection
  9. Pin SSMS to taskbar
  10. View Registered Servers (Ctrl + Alt + G)
  11. Select properties on SSDB00
    • Server name: (local)\SSDB00
    • Trust server certificate
    • Registered server name: SSDB00
    • Connection Properties tab…
    • Connect to database: tempdb
    • Connection time-out: 5 seconds
    • Use custom color: green
    • Save
  12. View Object Explorer Details (F7)

ODBC

If SQL Server ODBC 17 is already installed, it will most likely generate a error like "The feature you are trying to use is on a network resource that is unavailable.".
The solution is to uninstall SQL Server ODBC 17. This can be done by Control Panel > Remove Program.

SSDB Configuration

Query: ALTER DATABASE [model] SET RECOVERY SIMPLE WITH NO_WAIT;

Query:

EXECUTE master.sys.sp_configure
	@configname = N'remote access',
	@configvalue = N'0';
GO
RECONFIGURE WITH OVERRIDE;
GO

Query:

EXECUTE master.dbo.xp_create_subdir N'X:\MSSQL\Backup';

Query:

EXECUTE master.sys.sp_configure
	@configname = N'backup compression default',
	@configvalue = N'1';
GO
EXECUTE master.sys.sp_configure
	@configname = N'backup checksum default',
	@configvalue = N'1';
GO
RECONFIGURE WITH OVERRIDE;
GO
EXECUTE master.dbo.xp_instance_regwrite
	@rootkey = N'HKEY_LOCAL_MACHINE',
	@key = N'Software\Microsoft\MSSQLServer\MSSQLServer',
	@value_name = N'BackupDirectory',
	@type = N'REG_SZ',
	@value = N'X:\MSSQL\Backup';
GO

Query:

EXECUTE master.sys.sp_configure
	@configname = N'show advanced options',
	@configvalue = N'1';
RECONFIGURE WITH OVERRIDE;
GO
EXECUTE master.sys.sp_configure
	@configname = N'cost threshold for parallelism',
	@configvalue = N'50';
EXECUTE master.sys.sp_configure
	@configname = N'backup compression default',
	@configvalue = N'1';
EXECUTE master.sys.sp_configure
	@configname = N'backup checksum default',
	@configvalue = N'1';
RECONFIGURE WITH OVERRIDE;
GO
EXECUTE master.sys.sp_configure
	@configname = N'show advanced options',
	@configvalue = N'0';
RECONFIGURE WITH OVERRIDE;
GO

AdventureWorks Installation

See SQLAdmin post on Restore AdventureWorks on SQL Server 2019.

;

Plan Explorer Installation

  1. Download Plan Explorer from www.solarwinds.com/free-tools/plan-explorer. This requires registration with e-mail and phone#.
  2. Start Plan Explorer
  3. Pin to Start

History

2024-10-23 – Details added.
2024-09-22 – Element on administrators added. And a few typhographic corrections.
2024-09-09 – Post created.

 

2024-07-03

Stored Procedure Guidelines (T-SQL)

Here are some personal guidelines on writing and using a stored procedure. They are mentioned in a somewhat random order. The list and each recommendation will most likely change over time with new experinces, feedback and other new knowledge.

SET NOCOUNT ON

… to avoid unnecessary activity and communication on row count.

SET QUOTED_IDENTIFIER ON

... to make the code a little more robust and . Also against injection. And still I would prefer to enclose object and other identifiers in square brackets. This I think makes the code more strict and easy to maintain with several developers and teams involved.

Use schema name

... on used tables. Put procedure in custom schema without data.

Use tables

... not views, other procedures and triggers. And only functions with careful big-O considerations

Use sets

... not cursors or sub-selects as SQL Server is set-based. Cursors are fine when on DB2 or Oracle. Sub-selects are often seen on SAS.

Use functional name

... not sp_ or other Hungarian naming. It doesn´t really matter what type the object is.

Use sp_executesql

... for dynamic SQL. Not EXECUTE. In general you should avoid dynamic SQL. This includes prepared statements.

Use specific columns

... only take data that are required. Network traffic is then reduced to the required amount.

Use order of columns

... as given in the tables. One table a time. Re-arranging the columns could add significant resource consumption on each call.

Use keys

... and indexes with care. Use index on foreign keys. Check for covering index. Index usage might change with change of data in used tables.

Use table variables

... not temporary tables. This will keeps activity in buffer and away from storage,

Be direct

... and avoid temporary cached or staged data. Do not use dirty reads!

Be short

... by short transactions. Avoid costly isolation levels such as Repeatble Read. Also you can save some network traffic on often used stored procedures by short names on the procedure and the parameters.

Use effective parameters

... and avoid parameter sniffing. Focus on parameters where the values change a lot. Convert these parameters to local variables when suitable.

Check values

... before usage. Especially parameter values! Do the value check inside the procedure before usage.

Use effective types

... and avoid custom types. Custom types will add resource consumption on resources. And in some situation increase blockings on metadata.

Use effective operators

... avoid costly operators such as LIKE. Be carefull with COALESCE, string operations and aggregated values. T-SQL is for finding and sorting data. Use other languages for complex calculations and text processing.

Use explicit type cast

... not indirect. And you are still somewhat in control.

Use EXISTS

... not COUNT. As this is more effective with T-SQL.

Use effective error handling

... by testing before exception handling with TRY - CATCH. Consider exception handling as last-line-of-defence after tests and checks. Even when using exception handling this should be kept inseide the procedure. Throwing a exception outside the procedure could be rather costly on the resource consumption.

Use MS_Description

... extended property. This helps development with Visual Studio, see SQLAdmin on Descriptions.

Document

... procerdure, each parameter and output/return value in code, detailed and with examples. Also when you are the one and only developer.

Set security explicit

... on procedure through custom database roles on Active Directory groups. Using default database roles is outside your control.

Use ALTER PROCEDURE

... not DROP - CREATE PROCEDURE. Security and other external configurations will be lost on each change.

Monitor

... usage and performance with DMVs, XEvents etc. See Monitoring Performance of Natively Compiled Stored Procedures.

Call procedure effective

... with CommandType.StoredProcedure and SqlParameter objects on the SqlCommandobject. Use careful crafted connection string with secure authentication (SSPI) like Kerberos.

Reference

Kimberly Tripp, SQLSkills: Building High Performance Stored Procedures.

 

Top 10 stored procedure performance tuning tips in SQL server

2024-05-13

Buffer Pool scan

In a SQL Errorlog we saw a lot of events like this.

Buffer Pool scan took 10 seconds: database ID 6, command 'CHECKPOINT', operation 'FlushCache', scanned buffers 341183509, total iterated buffers 345933594, wait time 75 ms. See 'https://go.microsoft.com/fwlink/?linkid=2132602' for more information.

The refered article "Operations that trigger a buffer pool scan may run slowly on large-memory computers" states that

Prior to SQL Server 2022, there was no way to eliminate this problem.

But then another event came around where we looked at Indirect Checkpoint configuration. It turned out that the database originally was so old that the "Target Recovery Time" was 0 (zero).

When we changed this configuration to 60 then the Buffer Pool scan messages no longer showed up in the SQL Errorlog.

Another benefit was that spikes and average on Disk Write Latency were reduced significantly.

Kendra Little wrote some time ago a useful article (link) with good references and details when Target Recovery Interval changed default value.

2024-01-28

Integrated Security = SSPI

 The term Integrated Security = SSPI is a element in a connection string for an application to get access to a data source using a OLEDB or SqlClient provider. Usually it is just used because it is recommended by someone trustworthy.

SSPI is an abbreviation to Security Support Provider Interface which is a general implementation of the Generic Security Services Application Program Interface (GSSAPI). The interesting thing is that SSPI makes it possible to get the most secure authentication in a given context. In Windows it is at least a Kerberos or NTLM authentication. When possible you will get the more secure - and faster - Kerberos authentication. This is done by a negotiation, where some parameters like rank can be configured in Windows. The Windows configuration can be through Group Policies, both local and domain.

In relation to SSPI Kerberos and NTLM are Security Support Providers (SSPs) speaking Windows lingo. The documentation from Microsoft says that you can write our own SSP, but I couldn't find many details on the subject.

Speaking about Microsoft documentation the SSPI documentation as spred across Windows Server documentation on Windows Authentication with "Security Support Provider Interface Architecture" and Windows App Development on Remote Procedure Call (RPC) with "Security Support Provider Interface (SSPI)". In general I find it somewhat difficult to find documentation on the involved components in a login and authentication process.

Using SSPI is done with different parameters and values on each driver or provider.
With the SqlClient the keyword to the ConnectionString property is "Integrated Security" and the value is "SSPI" - which is „strongly recommended“ by Microsoft. Please notice that with SqlClient 5 the namespace is changed from System.Data.SqlClient to Microsoft.Data.SqlClient as the SqlClient component is removed from the .NET framework and is now available as a NuGet package.
With the ODBC driver the keyword is "Trusted_Connection" and the value is "Yes" which is an example on the big variations on both keywords and values. If you are using a driver from another vendor than Microsoft or a driver for another environment you should be very aware on these details as they can really bite you.

 

2024-01-23

PowerShell SMO by SqlServer module

Install SqlServer module:

Install-Module -Name sqlserver

Typical warning:

Untrusted repository
You are installing the modules from an untrusted repository. If you trust this repository, change its
InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from 'PSGallery'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): y


See installed module:

Get-Module -Name SqlServer -ListAvailable


Import SqlServer module in PowerShell session:

Import-Module -Name SqlServer


Create SMO Server object:

$smoServer = New-Object Microsoft.SqlServer.Management.Smo.Server '(local)\SSDB2019'

See SMO Server object information:

$smoServer.Information

2024-01-01

Top-ten posts 2023

2023 was – again – a year with far less posts than planned. Fortunately some elder posts still generate some activity on this blog.

The top-ten 2023 post list is something like Top-ten posts 2022.

RankMoveRank 2022TitleViewsCreated
101SqlBulkCopy with PowerShell1.9502017-07-30
2>9>10High performance script timer8142020-01-03
3+14ISO 8601 date formatting using PowerShell4792008-08-14
4>7>10ISO 4217 Currency list4252012-06-26
5+38SQL Server Agent schedule owner3602010-02-08
6>5>10Team Foundation Server on a single server3502018-09-19
7-16DBCC CHECKDB with PowerShell3062014-10-09
8>3>10Start shared HTA with PowerShell2712012-09-13
9>2>10VLF count2472011-01-11
10010Audit Log for Analysis Services2042014-01-08

This year there are several new items on the Top-ten. My personal biggest surprise is the item on ISO 4217.
But – still my favority ISO is 8601. The oldest post is more than ten years old, which is I find both interesting and disturbing.

The posts in this blog are personal notes rather than public presentations. This is why I am fine with the low number of views.

2023-02-14

Add SQL Agent job step with tokens in output file name

 If you create a new SQL Agent job step with tokens in output file name using the stored procedure sp_add_jobstep or want to add a filename with tokens to an existing job step using the stored procedure sp_update_jobstep you will get a error like

A fatal scripting error occurred.
Variable SQLLOGDIR is not defined.

In this case I was trying to use the token SQLLOGDIR, but it does not matter which token.

Many users of Ola Hallengren (LinkedIn) Maintenance Solution (link) had similar errors. Actually Ola is aware of the challenge - he just forgot to promote his solution ;-)

When you use the stored procedure sp_add_jobstep you with that call the internal stored procedure sp_add_jobstep_internal. You can see the definition of both procedures in the system database msdb with SQL Server Management Studio (SSMS).

The root-challenge is that somewhere along the path from calling sp_add_jobstep to setting the output file name on the job step the content of the parameter is interpreted. But at that point the execution is outside SQL Agent, and the tokens are not available.

The trick - that Ola forgot to promote - is to build a string with each token element seperated.

DECLARE @_token_log_directory NVARCHAR(MAX) = N'$' + N'(ESCAPE_SQUOTE(SQLLOGDIR))';
DECLARE @_token_job_name NVARCHAR(MAX) = N'$' + N'(ESCAPE_SQUOTE(JOBNAME))';
DECLARE @_token_date NVARCHAR(MAX) = N'$' + N'(ESCAPE_SQUOTE(DATE))';
DECLARE @_token_time NVARCHAR(MAX) = N'$' + N'(ESCAPE_SQUOTE(TIME))';
DECLARE @_output_file_name NVARCHAR(MAX) = @_token_log_directory + N'\' + @_token_job_name +  N'_' + @_token_date + N'_' + @_token_time + N'.txt';

This string can then be used as parameter value to the parameter @output_file_name in the stored procedures to a SQL Agent job step.

EXECUTE msdb.dbo.sp_update_jobstep @job_name=N'IndexOptimize - Db1.Schema1.Tbl1',
  @step_id=1,
  @output_file_name= @_output_file_name;

If you dig into Olas code to install the maintenance solution and search for the string „@TokenLogDirectory“ you will see that Ola build this string element by element. With careful seperation of $-sign and token names.

2023-01-27

UDL-file connection to SQL Server

UDL is a component in Windows named Universal Data Link. To be more specific it is a OLE DB tool that is a connection defined in a UDL-file a with a GUI to edit the connection.

The file is usually crated as a empty file with the type ".udl".

The GUI has three tabs as follows:

Provider

A list of local available OLE DB providers for various sources like SQL Server components. The default provider is "Microsoft OLE DB Provider for SQL Server". But the provider "Microsoft OLE DB Driver for SQL Server" is the latest SQL Server provider which I will continue with here as I would recommend using it.

The provider "SQL Server Native Client" should be avoided as it is deprecated.

Connection

Basic connection configuration points are available here for different types of connection on the given provider. Some typical configuration points are:

  • Server name: The server name can be short or long (FQDN) and can also be with protocol and port information. If you try to select a server the UDL GUI will try to scan the network for a server.
  • Log on: The default is unfortunately "SQL Server Authentication" but I would recommend "Windows Authentication" in general. This also has a field for Server SPN if you want Kerberos authentication.
    There are other Active Directory log on that could be relevant to more special situations. These are not available with other providers.
  • Database: A drop-down filed where a database can be entered or selected. If you want to select a database the UDL GUI will try to connect to the server.
    A more special situation is the possibility to connect to a database file as a attached database. This is sometimes used in more dynamic development situations.
Also there is the button "Test Connection" where the text explains quite well what the button does.

Advanced

This tab will only show some other configuration points. The three tabs together will not show all configuration points.

All

This is the access to all configuration points on the given provider. Some point for timeout, encryption or Application Name are only available in UDL GUI here.

Edit file

After entering the values and saving them by clicking OK you can edit the UDL-file with a text editor. This way you can see a working example of a connection string with the given provider and configuration values. This can be quite handy when working on a effective connection string or a string with specific features.

Reference

Microsoft Docs: "Universal Data Link (UDL) configuration"

Microsoft Docs: "Microsoft OLE DB Driver for SQL Server"

2022-11-29

MSDTC Operations

Microsoft Distributed Transaction Coordinator, abbreviated MSDTC or just DTC, is a somewhat aged but still very essential technology from Microsoft.

As a SQL Server DBA I have primarily met MSDTC under the Linked Server functionality. Here MSDTC serves the technical transactions between the SQL Server installation where the linked server is defined and the target of the linked server on another Windows Server. The MSDTC usage is implicit when using a linked server, primarily as the linked server is based on the OLEDB technology.

As a side note this is why you will often see OLEDB waits in SQL Server when working over a linked server.

Graphical User Interface

The primary GUI for MSDTC is the – also – elder Microsoft Management Console (MMC) component named "Component Services". This is a COM+ snap-in for MMC. If you are using a Danish Windows then the component is named "Komponenttjenester". It is not easy to find in-depth documentation on COM+ or MMC, and the voluminous (five-volume) "COM+ Developers´s Reference Library" is unfortunately no longer available in the book stores, that is either as new or used.

Usually you are working on the local MSDTC. You might find the path in Component Services long, but here it is:

  1. Console Root
  2. Component Services
  3. Computers
  4. My Computer
  5. Distributed Transaction Coordinator
  6. Local DTC

When you right-click Local DTC you have access to the properties of the Local DTC with three configuration areas in separate tabs. The first tab is on MSDTC tracing configuration.

The second tab is on MSDTC logging configuration.

The third and last tab is on MSDTC security configuration.

All the examples are with default values.

When expanding Local DTC you see two items:

  • Transaction List
  • Transaction Statistics

The first item Transaction List will give you a simple list of active transactions. But there is not much more you can see on each transaction or the transaction history.

The second item Transaction Statistics give you a rather simple graphical presentation of a few measures. There are no possibilities to drill-down or get other other details.

I think that the limited monitoring shows the age of the MSDTC component.

MSDTC Log Files

Is a standard Windows installation the MSDTC log files are in

C:\Windows\System32\MsDtc

The log files are in a proprietary binary Microsoft format, with no local tools to read the log files.

You can open the MSDTC log files with the command line tool tracefmt.exe. But you will have to download Windows SDK to get this tool. With Windows SDK you will get tracefmt.exe in both 32- and 64-bit editions in these (default) locations:

  • C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64
  • C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86

The paths are examples from the current Windows SDK version.

Using tracefmt.exe is not complicated but also not just point-and-click. Read the documentation and spend some time with the syntax to get a output you can use.

MSDTC Application Error

Some errors will surface in Windows Application Event Log like this example.

Log Name: Application
Source: Microsoft-Windows-MSDTC Client 2
Date: 28-11-2022 07:34:19
Event ID: 4879
Task Category: CM
Level: Warning
Keywords: Classic
User: N/A
Computer: SQLDB42.sqladmin.lan
Description:
MSDTC encountered an error (HR=0x80000171) while attempting to establish a secure connection with system SQLDB666.

MSDTC in AlwaysOn Availability Groups

As SQL Server AlwaysOn Availability Groups are based on Windows Failover Cluster, among others, you will have to go through some configuration details to get a robust transaction even during a failover. There are some basic documentation like Configure distributed transactions for an AlwaysOn availability group, but I really think you will have to do some more reading, build a sandbox and try several solutions thoroughly before going live.

This could be important if you are working wrestling with an application build with a high-level framework like .NET Entity Framework or Java Hibernate, as such frameworks tend to use client-side transactions and then rely on distributed transactions.

2022-08-10

SqlClient 5.0

The .NET data provider SqlClient for SQL Server has been released in version 5.0 for general availability. This is a new major version so you are advised to check the changes, especially the breaking changes.

At first there are added support for TDS 8.0, but that will not affect most developers or their products. Anyway there is a breaking change with this on the Encrypt property. A quick solution in many cases is to add the keyword TrustServerCertificate with the value true to the connection string. This will look like this:
…Integrated Security=SSPI;TrustServerCertificate=true;…
Using TrustServerCertificate is also described in the Microsoft documentation „Connection String Syntax“ in the section „Using TrustServerCertificate“.

But the added support for SPN on server and failover will be interesting to many working on critical systems using SQL Server AlwaysOn Availability Groups and Kerberos authentication.

I think that the most significant change is the breaking change on added dependency on the Microsoft.SqlServer.Server package. You should really take a detailed look on this if you are working with SqlClient based systems. Even if you are not a developer or architect but "only" Database Administrator or DevOps Engineer. The migration from the namespace Microsoft.Data.SqlClient.Server in the .NET framework to the namespace Microsoft.SqlServer.Server in the SqlClient package will require some sort of refactoring code.

See more in the blog post "Released: General Availability of Microsoft.Data.SqlClient 5.0" and the Release Notes on GitHub.

History

2022-08-10 : Post created.
2024-01-20 : Update with TrustServerCertificate.

2022-07-15

Execute on a subset of databases

Sometimes it is necessary to execute a command or a statement on a subset of the databases in a instance. A quick solution is to generate the commands in a SELECT on sys.database and then execute the output.

But when this has been done a few times for a recurring task the solution becomes boring and tiresome. And then a more automated solution is required.

Automating the entire statement build and execxution over a cursor and using sp_executesql looks like a nice T-SQL solution.

DECLARE @DatabaseID AS INT;
DECLARE @DatabaseName AS NVARCHAR(50);
 
DECLARE @DatabaseCursor AS CURSOR;
SET @DatabaseCursor = CURSOR LOCAL FAST_FORWARD FOR
  SELECT name, database_id
  FROM sys.databases
  WHERE databases.name LIKE 'wso2am!_%' ESCAPE ('!');
OPEN @DatabaseCursor;

FETCH NEXT FROM @DatabaseCursor INTO @DatabaseName, @DatabaseID;
WHILE @@FETCH_STATUS = 0
BEGIN
  DECLARE @BackupSql NVARCHAR(4000) = N'BACKUP DATABASE [' + @DatabaseName + N'] TO DISK =''R:\BACKUP\' + @DatabaseName + '.bak'' WITH COPY_ONLY;';
  DECLARE @Msg NVARCHAR(2047) = CONVERT(nchar(27), SYSUTCDATETIME()) + N'Z  Backup [' + @DatabaseName + N']...'
  RAISERROR (@Msg, 0,0) WITH NOWAIT;
  DECLARE @BeginTime DATETIME2 = SYSUTCDATETIME();
  EXECUTE master.dbo.sp_executesql @stmt = @BackupSql;
  DECLARE @EndTime DATETIME2 = SYSUTCDATETIME();
  SET @Msg = N'  Duration = ' + CAST(DATEDIFF(s , @BeginTime, @EndTime) AS NVARCHAR(42)) + N's';
  RAISERROR (@Msg ,0,0) WITH NOWAIT;

  FETCH NEXT FROM @DatabaseCursor INTO @DatabaseName, @DatabaseID;
END
 
CLOSE @DatabaseCursor;
DEALLOCATE @DatabaseCursor;

The example above is part of cloning a subset of databases. I hope that you can use this in other situations.

2022-07-11

RAISERROR with UNIQUEIDENTIFIER

The T-SQL type Uniqueidentifier is not a formatting possibility for at RAISERROR message. But converting the value to a string is a quick fix. This example is based on the Uniqueidentifier output from the procedure sp_add_jobschedule.

DECLARE @_schedule_uid UNIQUEIDENTIFIER = N'7f6a5df0-a68f-4b07-992f-b7595958061f';
DECLARE @_schedule_code NVARCHAR(50) = CONVERT(NVARCHAR(50), @_schedule_uid);
RAISERROR ( N'Schedule UID = %s', 0,0, @_schedule_code ) WITH NOWAIT;

The output is then

Schedule UID = 7F6A5DF0-A68F-4B07-992F-B7595958061F

A small detail is that with the conversion the value is also converted to upper-case.

2022-05-31

Descriptions

Databases and objects in databases on SQL Server can be described with the Extended Property MS_Description. This description can be read by and used with Visual Studio, which gives a handy access to detailed documentation.

Unfortunately does instances and server-level objects like logins not have Extended Properties. So these elements can not have a description direct attached. There are some alternatives you can consider for these elements such like custom registry keys, file in root directory. But please don't create a custom table in the master database!

Description Lifecycle

A description can be created with the stored procedure sp_addextendedproperty. The procedure has eight parameters to control the name, value and level in the object hierarchy of the extended property. The value of the description has the type sql_variant, and the size can not be more than 7,500 bytes.

Later the description can be changed with the stored procedure sp_updateextendedprocedure.

Sometimes a tool or a clone can require the description to be removed. This can be done with the stored procedure sp_dropextendedproperty.

Extended Properties can be viewed through the catalog view sys.extended_properties or the function sys.fn_listextendedproperty.

Example on table description
EXECUTE sys.sp_updateextendedproperty
  @name=N'MS_Description', @value=N'List of SQL Server collations',
  @level0type=N'SCHEMA',@level0name=N'sqladmin',
  @level1type=N'TABLE',@level1name=N'collation';

SQL Server Management Studio

In SQL Server Management Studio (SSMS) the description on a object can be set as described above. But there are two more ways to add a description to a object. They are short described below, but the order they are mentioned in does not imply any ranking of the possibilities.
The first way to add a description is in the SSMS table design tool. Right-click on a object in the Object Explorer and click Design. Show the properties of the object with another right-click or press Alt+Enter. Then you can edit the Description property.



The second way to add a description is in the Database Diagrams of a given database. In here you can access the properties of a objects and edit the Description property.

Visual Studio

In SQL Server Management Studio (SSMS) extended properties are in the object properties. Here each extended property like MS_Description can be created, viewed, changed and deleted.

A Visual Studio data project can be created on a existing database or with a new database. Open the design of a table and the description in MS_Description is in the properties window, usually down to the right in Visual Studio.


In the example above the table "collation" has the description "SQL Server collations", that is placed in the extended property MS_Description of the table.

Actually the value of the description can also be handled (create/change) in a Visual Studio data project.

Data Classifications

When Microsoft introduced data classification with SQL Server 2012 this information was stored in extended properties. But with SQL Server 2016 this information is in (hidden) system tables.

Discussion

I can only recommend that you put the extended property MS_Description on each database and every object in the databases. At first it wil be a disturbance and somewhat annoying. But when you databases get in production, have lived for a few years and new developers og DBAs are to keep the database alive, then you (hopefully) will experience the benefits. That be in SSMS, Visual Studio or a 3rd part tool.

As a start you could add MS_Description to your static code test. At least on database, data objects and functional object. Later you can extend the test to cover more technical parts of the database like filegroups or users.

History

2023-01-01 : SSMS section added.
2022-05-31 : Entry created.

2022-05-05

Restore AdventureWorks

There are several SQL Server sample databases from Microsoft, and one of the easiest to start and still somewhat up-to-date is AdventureWorks. There are various versions and editions of the database, and they are described in the Microsoft document "AdventureWorks sample databases".

This is an example on how to restore AdventuresWorks on a SQL Server Database Engine instance. The scripts and technical details are in the Github repository SQLAdmin (AdventureWorks).

Before you start restoring you have to make the backup file available to the instance. This can be done by copying the file to the default backup location of the Database Engine instance.

Then you can start the restore and some minor configurations to make the database available as below.

The configurations are not necessary to make the database available to you, but they are nice to make it general available and functional on SQL Server. Configuring Query Store on the database is optional before SQL Server 2022, but could be handy on a non-production instance.

History

2024-10-28 : Post made general and with scripts in Github.
2022-05-05 : Post created on SQL Server 2019

2022-03-25

xp_dirtree (T-SQL)

It is possible to get the content of a directory from T-SQL with the Extended Stored Procedure (XP) master.dbo.xp_dirtree. This only requires the SQL Server service account to have access to the directory and have browse rights.

Syntax:

EXECUTE master.dbo.xp_dirtree <Directory>, <Depth>, <IsFile>

  • <Directory> nvarchar(???): Full path (name) of directory. Can be UNC path. The value can end with '\' or not.
  • <Depth> int: Depth of list in directory.
  • <IsFile> int: Show if row is on a file. If the value in non-zero then the result will show if a row is in a file.

The result is a table with the three columns subdirectory, depth and file where

  • subdirectory is the name of the subdirectory or file in the given directory.
  • depth is the depth of the directory/file in the given directory. One ('1') is the root of the current directory.
  • file indicates if the directory is a file ('1') or not ('0'). If not a file then it is a directory.

For some reason the value for <Directory> can't be from SERVERPROPERTY like
EXECUTE master.dbo.xp_dirtree SERVERPROPERTY('InstanceDefaultBackupPath'), 1,1;.
The error is
Msg 102, Level 15, State 1, Line 1
Incorrect syntax near 'InstanceDefaultBackupPath'.

Not even with CONVERT like
EXECUTE master.dbo.xp_dirtree CONVERT(nvarchar(512), SERVERPROPERTY('InstanceDefaultBackupPath')), 1,1;
where the error is
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'CONVERT'.

The system function SERVERPROPERTY can be used through a temporary variable like this
DECLARE @_backuppath nvarchar(512) = CONVERT(nvarchar(512), SERVERPROPERTY('InstanceDefaultBackupPath'));
EXECUTE master.dbo.xp_dirtree @_backuppath, 1,1;

Example:

EXECUTE master.dbo.xp_dirtree N'D:\MSSQL\Backup', 1, 1;
List (backup) files in folder.

Unfortunately this Extended Stored Procedure is not documentd by Microsoft. Like many other XP's...

History

2024-06-25: Examples on SERVERPROPERTY added.
2022-03-25: Post published.

2022-01-01

Top-ten posts 2021

Looking back at 2021 not only as another year with the Covid-19 pandemic. For various reasons it has only been a year with fewer posts than planned. 

The top-ten from 2021 is very much like the top-ten list from 2020 (link).

  1. SqlBulkCopy with PowerShell (2017-07-30)
  2. ISO 8601 date formatting using PowerShell (2008-08-14)
  3. DBCC CHECKDB with PowerShell (2014-10-09)
  4. xp_instance_regwrite syntax (2013-09-10)
  5. T-SQL formatted duration (2017-04-29)
  6. Audit Log for Analysis Services (2014-01-08)
  7. Start shared HTA with PowerShell (2012-09-13)
  8. SQL Server Agent schedule owner (2010-02-08)
  9. PowerShell MessageBox (2014-03-15)
  10. Windows Storage Spaces (2019-03-27)
The date on each post is the creation date. Some posts I have updated since the creation, and then the history is described in the post itself. The oldest post is more than ten years old, and that I personally find quite satisfying.

Still I do not have many readers, That is fine with me as the purpose of this blog is more a personal collection of refined notes. Looking at the number of views on each post there is a rather big difference. The #1 post has about 20 times more views than post #10.

2021-02-08

SQL Server troubleshooting

Microsoft has moved the SQL Server troubleshooting documentation to a new and dedicated location:

https://docs.microsoft.com/en-us/troubleshoot/sql/welcome-sql-server

The documentation is moved from Knowledge Base articles. These articles still exists but are now focused on release notes.

See more details on the new SQL Server trouble shooting dokumentation site:

https://techcommunity.microsoft.com/t5/sql-server-support/new-home-for-sql-server-troubleshooting-kb-articles-and-for/ba-p/2113331