SQL Lab shows "The database that was used to generate this query could not be found" for most users when there are > 20 databases

[hf-13]

Screenshots/recordings

No response

Superset version

6.1.0

Python version

3.9

Node version

16

Browser

Chrome

Additional context

Superset version

6.1.0

What's the issue?

When SQL Lab initializes, it fetches the database list via:
GET /api/v1/database/?q=(filters:!((col:database_name,opr:neq,value:examples)))

No page_size is specified, so FAB defaults to 20 results. The response is dispatched as SET_DATABASES, populating the Redux databases map with only those 20 entries (keyed by integer id).

When a user's saved tab has a database_id that doesn't appear in the first 20 results, databases[dbId] is undefined -> the frontend evaluates F = isEmpty(databases[dbId]) as true -> the yellow warning banner appears:

The database that was used to generate this query could not be found
Choose one of the available databases on the left panel.

The database exists, permissions are correct, and the tab state in the backend is valid. This is a pure frontend state population problem. Reloading the page sometimes resolves it (timing-dependent). Clicking the database in the left panel always resolves it (that search uses page_size=1000 and correctly finds the database).

Steps to reproduce

1. Have a Superset instance with more than 20 databases
2. Ensure a user has a saved SQL Lab tab whose database_id is not among the first 20 results of GET /api/v1/database/ (default sort)
3. Open SQL Lab
4. Observe the yellow "database not found" banner on that tab despite the database being visible and selectable in the left panel

Expected behavior

SQL Lab loads without the "database not found" error. All saved tabs display correctly regardless of how many databases exist.

Actual behavior

Any user whose saved tab references a database outside the first 20 API results sees the error on every fresh SQL Lab load.

Root cause

GET /api/v1/database/ without an explicit page_size returns FAB's default of 20 rows. The SQL Lab init code does not request a larger page size or paginate through results. The Redux databases map is
therefore incomplete.

Mainly - SQL Lab entry bundle, SET_DATABASES reducer.

Workaround

Patch DatabaseRestApi.page_size at app startup via FLASK_APP_MUTATOR:

def FLASK_APP_MUTATOR(app):
    from superset.databases.api import DatabaseRestApi
    DatabaseRestApi.page_size = 100000

This causes the unpaginated init call to return all databases. DatabaseRestApi.max_page_size is already -1 (unlimited), so no server-side cap blocks this.

Proposed fix

Option A — explicitly request all databases
Option B — defer tab rendering until SET_DATABASES is fully populated, then fall back to a targeted fetch for any dbId still missing from the map.

Environment

• Superset 6.1.0
• ~885 databases in the instance
• FAB default page_size: 20
• DatabaseRestApi.max_page_size: -1 (unlimited)

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

I can confirm this bug and the root cause analysis is spot-on, though the exact mechanism is slightly different from what's described in the issue.

The actual flow:

1. SQL Lab bootstraps via GET /api/v1/sqllab/ which calls DatabaseDAO.find_all() — this fetches all databases without pagination [1] [2]. So the initial Redux state is complete.

2. Then SqlEditorLeftBar mounts and renders the DatabaseSelector component, which calls loadDatabases with pagination (page and page_size parameters) [3].

3. The paginated result (default ~20 databases) is passed to the handleDbList callback, which dispatches SET_DATABASES [4].

4. The SET_DATABASES reducer completely replaces the databases state rather than merging with the existing bootstrap data [5]:

   [actions.SET_DATABASES]() {
     const databases = {};
     action.databases.forEach(db => {
       databases[db.id] = { ...db, extra_json: JSON.parse(db.extra || '') };
     });
     return { ...state, databases };  // full replacement, not merge
   },

5. All databases from the bootstrap that weren't in the paginated response are now gone from Redux → the "database not found" warning fires [6].

The fix should target the overwrite, not the fetch. A few approaches:

• Change SET_DATABASES to merge instead of replace: return { ...state, databases: { ...state.databases, ...databases } }
• Remove the getDbList dispatch from SqlEditorLeftBar since the bootstrap already provides the complete list, and let DatabaseSelector manage its own local state for the dropdown
• Pass a large page_size in the loadDatabases call (your Option A), though this only masks the real issue

Your FLASK_APP_MUTATOR workaround is valid for now but won't prevent the overwrite — it works because it changes the default page size for all endpoints, which happens to affect the DatabaseSelector fetch too. The cleanest short-term fix would be changing the reducer to merge.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[GagandeepSingh20]

Hi, I'd like to take a look at this issue. I'll try to reproduce it locally and investigate the reducer/state update flow described above. If I can confirm the root cause, I'll work on a fix and open a PR.

[GagandeepSingh20]

Hi, I investigated the code path locally and traced the issue to the SQL Lab database state flow. DatabaseSelector fetches paginated database results and passes them to setDatabases(). The SET_DATABASES reducer currently rebuilds and replaces the entire state.databases map, which appears to remove previously loaded database IDs from Redux. This seems consistent with the reported behavior. I'll work on a fix and add a regression test.

[rusackas]

Thanks for opening a PR. Maybe @sadpandajoe will want to take a look at this...