Skip to content

fix(json): make cjson instances inherit APISIX's cjson options#13680

Merged
AlinsRan merged 4 commits into
apache:masterfrom
AlinsRan:chore/bump-lua-resty-session-api7
Jul 15, 2026
Merged

fix(json): make cjson instances inherit APISIX's cjson options#13680
AlinsRan merged 4 commits into
apache:masterfrom
AlinsRan:chore/bump-lua-resty-session-api7

Conversation

@AlinsRan

@AlinsRan AlinsRan commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #13440

lua-cjson options are per-instance, and cjson.new() starts from the compile-time defaults rather than inheriting whatever was set on the module singleton. core/json.lua only configures the cjson.safe singleton, so every dependency holding a private instance escapes itlua-resty-session, lua-resty-healthcheck, lua-resty-worker-events and lua-resty-aws all keep one, and the plain cjson singleton is a separate config that had no options set either.

That is what corrupts OIDC userinfo. openid-connect stores userinfo in the session and reads it back on later requests:

step instance result
openidc decodes userinfo cjson.safe singleton roles gets array_mt
session save (encode) lua-resty-session private instance stored as []
session load (decode) lua-resty-session private instance array_mt lost
core.json.encodeX-Userinfo singleton {}

The array gets in but cannot get out: array_mt is honoured on encode by any instance, but whether decode attaches it depends on that instance’s own flag.

Fix

Set the options on both cjson singletons and wrap cjson.new / cjson.safe.new so every instance created afterwards inherits them. init.lua runs patch() before any dependency is loaded, and these instances are created lazily on first use, so the wrapped factory catches all of them. Only the default changes — a library that sets an option explicitly on its own instance still wins.

This replaces the earlier api7-lua-resty-session dependency swap: rather than forking each escaping library in turn, it fixes the whole class, and the next dependency with a private instance just works.

encode_escape_forward_slash is set too. It looks redundant today because in the bundled lua-cjson (2.1.0.11) that setter mutates a shared escape table, so setting it anywhere already applies process-wide — I verified an instance created before the call is retroactively affected. But upstream has made it per-instance (openresty/lua-cjson@fa418dc, 2026-06-14), so once a runtime upgrade picks that up, every private instance would silently go back to escaping /. Setting it here pins the current behaviour instead of relying on that implementation detail.

Approach suggested by @nic-6443.

Verification

With the stock lua-resty-session (no fork), against the real test-nginx harness:

  • t/core/json.t — new TEST 9/10 cover both options across the cjson.safe instance, the plain cjson instance and the plain singleton.
  • t/plugin/openid-connect-userinfo-array.t — plugin-level e2e: forges an authenticated session with the plugin secret, drives a request through routing → openid-connect → upstream (which echoes the headers it received), and asserts the empty roles claim arrives in X-Userinfo as []. No IdP needed: the fix only manifests on the session load path, which every request after the callback takes.

Both fail against the unpatched tree (got: {} / expected: []) and pass with the patch.

Note on scope

array_mt now attaches to arrays decoded by every dependency. The hazard to watch for in a full-suite run: code that decodes a JSON array (especially an empty one) and then assigns string keys to that same table — it used to re-encode as an object, and would now encode as [], silently dropping the keys. Read-only use (pairs, #, next) is unaffected.

…serinfo

The shared cjson instance in lua-resty-session decodes JSON without the
array metatable, so an empty array loses its array identity across the
session save/load cycle and is re-encoded as an object. With the
openid-connect plugin this turns empty userinfo claims such as `roles`
into `{}`, and `X-Userinfo` carries `"roles":{}` instead of `"roles":[]`.

Switch to api7-lua-resty-session 4.1.6-0, which enables
decode_array_with_array_mt on that instance.

lua-resty-openidc and lua-resty-saml still depend on the upstream
lua-resty-session, so it is still pulled in transitively, but
api7-lua-resty-session sorts before it and therefore owns the
resty.session.* module paths -- the same arrangement already used for
api7-lua-resty-http and api7-lua-resty-jwt.

Fixes apache#13440
@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. dependencies Pull requests that update a dependency file labels Jul 9, 2026
…nd-trip

Reproduces the apache#13440 chain without an IdP: core.json.decode (as openidc
parses userinfo) -> session encode/decode -> core.json.encode (as the plugin
builds X-Userinfo). With the old lua-resty-session the last step yields
`{}`; the test therefore fails if the dependency is reverted.
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jul 9, 2026
…ycle

Replace the direct session_utils check with an end-to-end test: a request
is routed through the openid-connect plugin to an upstream that echoes the
headers it received, and the test asserts the empty `roles` claim arrives
in X-Userinfo as `[]`.

The session is forged with the plugin's own secret so the plugin takes the
already-authenticated path and restores the userinfo from the session --
the same path every request takes after the callback, and the one where
the empty array used to be lost. A non-expired access token plus
renew_access_token_on_expiry=false keep openidc from reaching for the
discovery document, so no IdP is needed.

Against the old lua-resty-session the upstream receives `"roles":{}`.
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Jul 9, 2026
@nic-6443

Copy link
Copy Markdown
Member

Stepping back a bit: the root cause here is a pattern, not just lua-resty-session. lua-cjson options are per-instance, and cjson.new() doesn't inherit whatever we set on the singleton — it always starts from the compile-time defaults. core/json.lua only flips decode_array_with_array_mt(true) on the cjson.safe singleton, so every dependency that creates a private instance escapes it. In the current deps tree that's at least lua-resty-session, lua-resty-healthcheck, lua-resty-worker-events and lua-resty-aws, and the plain require("cjson") singleton is a separate config that doesn't have the option either.

So instead of chasing each library one by one (fork, pin, re-audit every new dependency), we could fix this globally in apisix/patch.lua: set the option on both singletons and wrap cjson.new / cjson.safe.new so every instance created afterwards gets the same default. init.lua calls require("apisix.patch").patch() before any plugin or dependency is loaded, so the ordering is safe. And since this only changes the default, a library that explicitly calls decode_array_with_array_mt(false) on its own instance still wins.

Quick check that the wrap works:

local cjson_safe = require("cjson.safe")
local orig_new = cjson_safe.new
cjson_safe.new = function(...)
    local inst = orig_new(...)
    inst.decode_array_with_array_mt(true)
    return inst
end

-- later, inside a dependency:
local dep = require("cjson.safe").new()
dep.encode(dep.decode('{"items":[]}'))  --> {"items":[]}

The trade-off is that it's a behavior change for all decoded arrays across every dependency (they now carry cjson.array_mt), so it deserves a full test-suite run rather than a targeted regression test. But it would let us drop the api7-lua-resty-session fork later, and the next dependency with a private cjson instance just works.

The upstream fix (bungle/lua-resty-session#207) is still worth pursuing either way. What do you think? Happy to submit a PR for the patch.lua approach if this direction makes sense.

@AlinsRan

AlinsRan commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

Agreed — your framing is the right one, and it supersedes this PR.

I verified your claims against the current deps tree:

A) singleton has array_mt set, does .new() inherit it?   false
B) does require("cjson") get it too?                     false   (separate config)
C) unpatched lua-resty-session + the cjson.safe.new wrap:
     {"sub":"a UID","roles":[]}

(C) is the one that settles it — with the wrap in place, the stock lua-resty-session already round-trips correctly, so the fork buys nothing. And the private-instance list matches what you found: resty/session/utils.lua, resty/healthcheck.lua, resty/worker/events.lua, resty/aws/raw-api/*.lua. Ordering is fine too: init.lua:28 runs patch() before any dependency is loaded, and these instances are created lazily on first use, so the wrapped factory catches all of them.

@AlinsRan AlinsRan changed the title chore(deps): use api7-lua-resty-session to fix empty arrays in OIDC userinfo fix(cjson): make dependency-owned cjson instances keep empty arrays as arrays Jul 13, 2026
membphis
membphis previously approved these changes Jul 13, 2026
@nic-6443

Copy link
Copy Markdown
Member

One more data point on the patch.lua idea: core/json.lua sets a second option, encode_escape_forward_slash(false) (#8684), and it turns out that one is sitting on a semantic boundary in lua-cjson right now.

In the lua-cjson bundled by apisix-runtime (2.1.0.11), that setter mutates a shared static escape table, not per-instance config. I verified it: calling it on the cjson.safe singleton flips the behavior of previously-created private instances and of the plain cjson module too. So today core/json.lua already covers every dependency's private instance process-wide — by accident.

But upstream changed this on 2026-06-14 (openresty/lua-cjson: "setting encode_escape_forward_slash on a cjson instance does not affect other instances" — char2escape moved into the per-config struct). The moment a future runtime upgrade picks that up, every private cjson instance in our deps silently reverts to escaping / as \/. That's a byte-level output change riding along a runtime bump, which matters anywhere serialized JSON gets signed or string-compared.

So the patch.lua wrapper should set both options. For encode_escape_forward_slash(false) it isn't even a behavior change — it just pins today's actual process-wide behavior so it survives the upstream semantic change. It also aligns with basically every mainstream serializer: JS, Python, Go, Ruby, Java (Jackson/Gson), Rust, .NET all leave / unescaped; PHP is the lone holdout that escapes by default. Escaping / is a leftover from the embed-JSON-in-<script> era, and RFC 8259 treats both forms as equivalent anyway.

With that, patch.lua becomes the single source of truth for process-wide cjson defaults, and the two option lines in core/json.lua can simply be removed. I checked for blind spots: nothing under apisix/cli/ requires core.json, so there's no code path that loads it without going through init.lua's patch() first.

@AlinsRan AlinsRan dismissed stale reviews from shreemaan-abhishek and membphis via dece883 July 14, 2026 04:21
@AlinsRan AlinsRan force-pushed the chore/bump-lua-resty-session-api7 branch from 1ce0dd8 to dece883 Compare July 14, 2026 04:21
@AlinsRan AlinsRan changed the title fix(cjson): make dependency-owned cjson instances keep empty arrays as arrays fix(json): make cjson instances inherit APISIX's cjson options Jul 14, 2026
lua-cjson options are per-instance and cjson.new() starts from the
compile-time defaults, so enabling decode_array_with_array_mt on the
cjson.safe singleton in core/json.lua never reaches a dependency that
keeps a private instance. Those dependencies decode arrays without the
array metatable, and an empty array is re-encoded as an object.

Enable the options on both cjson singletons and wrap cjson.new /
cjson.safe.new so instances created afterwards inherit them. This replaces
the api7-lua-resty-session dependency swap: the fix now covers every
dependency with a private instance (lua-resty-healthcheck,
lua-resty-worker-events, lua-resty-aws, ...), not just lua-resty-session.

encode_escape_forward_slash is set as well. It looks redundant today because
the bundled lua-cjson mutates a shared escape table, so setting it anywhere
already applies process-wide -- but upstream has made it per-instance, and
once a runtime upgrade picks that up every private instance would silently
go back to escaping '/'. Setting it here pins the current behaviour instead
of relying on that implementation detail.

Fixes apache#13440
@AlinsRan AlinsRan force-pushed the chore/bump-lua-resty-session-api7 branch from dece883 to e345c6b Compare July 14, 2026 05:29

@membphis membphis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AlinsRan AlinsRan merged commit 6c7b56a into apache:master Jul 15, 2026
16 checks passed
@AlinsRan AlinsRan deleted the chore/bump-lua-resty-session-api7 branch July 15, 2026 05:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

help request: how to debug weird user info returned/altered in the OpenID connect plugin?

4 participants