[Source Versioning] SS-60 update docs for SQL server source versioning

[patrickwwbutler]

Updates docs describing process to add/remove columns from SQL Server upstream sources

Motivation

• This PR adds a known-desirable feature.
  https://linear.app/materializeinc/issue/SS-60/write-docs-for-sql-server-source-versioning

Checklist

• This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
• This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
• If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
• If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
• If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

[martykulma]

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

[patrickwwbutler]

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

CREATE SOURCE contains a "pointer" to the sql-server-considerations.md file, so it should receive this change notifying users of the source versioning workflow as well.

As for CREATE TABLE, it looks like we provide 2 different syntaxes, "Read/write table" and "PostgreSQL source table" https://materialize.com/docs/sql/create-table/

From what I can tell, isn't the syntax for creating a source table the same for each source? And it's only certain functionality that is actually dependent on the source type? Perhaps that section should just be renamed to "read-only source table"?

cc @aljoscha for some confirmation on my understanding of the mzsql syntax here.

[martykulma]

Should also update the SQL pages for CREATE SOURCE and CREATE TABLE to include SQL Server.

CREATE SOURCE contains a "pointer" to the sql-server-considerations.md file, so it should receive this change notifying users of the source versioning workflow as well.

As for CREATE TABLE, it looks like we provide 2 different syntaxes, "Read/write table" and "PostgreSQL source table" https://materialize.com/docs/sql/create-table/

From what I can tell, isn't the syntax for creating a source table the same for each source? And it's only certain functionality that is actually dependent on the source type? Perhaps that section should just be renamed to "read-only source table"?

cc @aljoscha for some confirmation on my understanding of the mzsql syntax here.

The syntax of CREATE TABLE .. FROM SOURCE is the roughly the same (I think it's the same for OLTP sources, Kafka is a little different). Renaming "postgres source table" sounds like the best approach. Just need to make sure to specify PG & SQL Server in the sections.

[kay-kim]

Could we actually create a separate tab for SQL Server? Reasons:

• SQL Server doesn't support TEXT COLUMNS, yes?
• Once we get more source versioning turned on, we'll actually separate out the tabs into separate pages ... so that people don't have to wade into the page for content relevant to their source.

[martykulma]

• SQL Server doesn't support TEXT COLUMNS, yes?

SQL Server does support text columns!

[kay-kim]

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)

CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]

and we also state that we don't support types text and ntext ...

[martykulma]

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)

CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]

and we also state that we don't support types text and ntext ...

whoops - should add TEXT COLUMNS there!

The unsupported types are correct. SQL Server CDC doesn't play well with text and ntext, and they are deprecated types. Customers can use varchar(max) / nvarchar(max)

[patrickwwbutler]

Oh ... then, FYI -- the current create source for sql server only lists EXCLUDE COLUMNS (...)

CREATE SOURCE [IF NOT EXISTS] <src_name>
[IN CLUSTER <cluster_name>]
FROM SQL SERVER CONNECTION <connection_name>
  [ ( EXCLUDE COLUMNS (<col1> [, ...]) ) ]
<FOR ALL TABLES | FOR TABLES ( <table1> [AS <subsrc_name>] [, ...] )>
[WITH (RETAIN HISTORY FOR <retention_period>)]

and we also state that we don't support types text and ntext ...

Do the TEXT COLUMNS and EXCLUDE COLUMNS clauses actually go in the CREATE SOURCE statement? I would have assumed that they go in CREATE TABLE FROM SOURCE statement. Even the syntax we have in the docs here doesn't really make sense, because there is no table specified for which to exclude/text columns from?

[kay-kim]

Do the TEXT COLUMNS and EXCLUDE COLUMNS clauses actually go in the CREATE SOURCE statement? I would have assumed that they go in CREATE TABLE FROM SOURCE
So. ...

• for people using the old syntax, it'd be in the CREATE SOURCE statement.
• for people using the new syntax, it'd be in the CREATE TABLE statement.