Skip to content

fix: convert tabs to spaces when stripping whitespace #831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RamiNoodle733 wants to merge 1 commit into
base: master
Choose a base branch
from
+3 −0
Open

fix: convert tabs to spaces when stripping whitespace #831

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions sqlparse/filters/others.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ def _stripws_default(tlist):
is_first_char = True
for token in tlist.tokens:
if token.is_whitespace:
# Replace tabs with spaces
token.value = token.value.replace('\t', ' ')
# Then normalize whitespace (collapse multiple spaces)
Comment on lines +95 to +97
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tab-to-space conversion on line 96 is ineffective because line 98 immediately overwrites the entire token value regardless of what line 96 does. Line 98 sets token.value to either an empty string or a single space, completely replacing whatever was set on line 96. This means the fix doesn't actually normalize tabs to spaces as intended.

To properly fix this issue, the tab-to-space replacement should either:

  1. Be done before whitespace tokens are processed by this filter, or
  2. Be integrated into the logic on line 98 so it's not immediately overwritten

However, it's worth investigating whether tabs in whitespace tokens are actually surviving the current code (line 98 should already normalize all whitespace to either empty string or single space). The issue description suggests tabs are still present after formatting, but the current logic on line 98 should already handle this without the change on line 96.

Suggested change
# Replace tabs with spaces
token.value = token.value.replace('\t', ' ')
# Then normalize whitespace (collapse multiple spaces)
# Normalize whitespace (collapse multiple spaces)

Copilot uses AI. Check for mistakes.
token.value = '' if last_was_ws or is_first_char else ' '
Comment on lines +95 to 98
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description indicates "Tests:" but no tests were added to verify this fix. Given that issue #823 specifically mentions that tabs are not being converted to spaces when reindent=True, a test should be added to reproduce the issue and verify the fix. For example, a test could format SQL containing tabs with reindent=True and assert that the output contains no tab characters.

Copilot uses AI. Check for mistakes.
last_was_ws = token.is_whitespace
is_first_char = False
Expand Down
Loading