Feature/mongo poc by jansigi · Pull Request #94 · ergon/dope-query-builder

jansigi · 2026-03-25T15:46:53Z

No description provided.

Copilot

Pull request overview

Adds an initial MongoDB “proof of concept” module to the project, implementing a QueryResolver that converts the existing Dope clause/expression model into MongoDB aggregation/update/delete representations, plus integration tests to validate behavior against a real Mongo instance.

Changes:

Add new mongo Gradle subproject and wire it into settings.gradle.kts.
Implement Mongo resolvers (ClauseResolver, ExpressionResolver, MongoResolver) and MongoDopeQuery result types.
Add Mongo integration tests using Testcontainers and the MongoDB Java driver; minor core/join API convenience addition.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
settings.gradle.kts	Includes the new `mongo` subproject in the build.
mongo/build.gradle.kts	Adds Mongo module build + integration test source set and dependencies.
mongo/src/main/kotlin/ch/ergon/dope/mongo/MongoDopeQuery.kt	Introduces Mongo-specific `DopeQuery` sealed types and `buildMongo` helpers.
mongo/src/main/kotlin/ch/ergon/dope/mongo/resolver/MongoResolver.kt	Entry-point resolver delegating to clause/expression resolvers.
mongo/src/main/kotlin/ch/ergon/dope/mongo/resolver/ExpressionResolver.kt	Renders Dope expressions into Mongo filter fragments.
mongo/src/main/kotlin/ch/ergon/dope/mongo/resolver/ClauseResolver.kt	Renders Dope clauses into aggregation/update/delete documents and lookup logic.
mongo/src/integrationTest/kotlin/ch/ergon/dope/mongo/integrationTest/TestMongoDatabase.kt	Spins up MongoDB container and seeds test data.
mongo/src/integrationTest/kotlin/ch/ergon/dope/mongo/integrationTest/BaseIntegrationTest.kt	Executes built queries against Mongo using `Document.parse`.
mongo/src/integrationTest/kotlin/ch/ergon/dope/mongo/MongoIntegrationTest.kt	End-to-end integration coverage for select/join/order/limit/update/delete.
couchbase/build.gradle.kts	Adds a new test dependency (`classgraph`).
core/src/main/kotlin/ch/ergon/dope/resolvable/clause/model/mergeable/MergeableClause.kt	Adds computed `onType` convenience property for join clauses.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T15:55:34Z

+                val fields = allVariables.joinToString(", ") { variable ->
+                    val value = if (variable.value is IField<*>) {
+                        "\"\$${(variable.value as IField<*>).name}\""
+                    } else {
+                        variable.value.toDopeQuery(this).queryString
+                    }
+                    "\"${variable.name}\": $value"
+                }
+
+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages + "{ \$addFields: { $fields } }",


LetClause resolution returns a new Aggregation without propagating parent.namedParameters, which drops any parameters accumulated earlier in the query chain. Pass through namedParameters = parent.namedParameters (and merge with any parameters produced while rendering variable values).

Suggested change

val fields = allVariables.joinToString(", ") { variable ->

val value = if (variable.value is IField<*>) {

"\"\$${(variable.value as IField<*>).name}\""

} else {

variable.value.toDopeQuery(this).queryString

}

"\"${variable.name}\": $value"

}

MongoDopeQuery.Aggregation(

stages = parent.stages + "{ \$addFields: { $fields } }",

var namedParameters = parent.namedParameters

val fields = allVariables.joinToString(", ") { variable ->

val value = if (variable.value is IField<*>) {

"\"\$${(variable.value as IField<*>).name}\""

} else {

val valueQuery = variable.value.toDopeQuery(this)

namedParameters = namedParameters.merge(valueQuery.namedParameters)

valueQuery.queryString

}

"\"${variable.name}\": $value"

}

MongoDopeQuery.Aggregation(

stages = parent.stages + "{ \$addFields: { $fields } }",

namedParameters = namedParameters,

Copilot · 2026-03-25T15:55:35Z

+            is UnnestClause<*, *> -> {
+                val parent = clause.parentClause.toDopeQuery(this) as MongoDopeQuery.Aggregation
+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages + "{ \$unwind: \"\$${clause.arrayTypeField.name}\" }",


UnnestClause resolution also drops parent.namedParameters when constructing the new Aggregation. This can lose parameters from earlier stages. Propagate namedParameters = parent.namedParameters here as well.

Suggested change

stages = parent.stages + "{ \$unwind: \"\$${clause.arrayTypeField.name}\" }",

stages = parent.stages + "{ \$unwind: \"\$${clause.arrayTypeField.name}\" }",

namedParameters = parent.namedParameters,

Copilot · 2026-03-25T15:55:35Z

+                    updateDocument = mergeUpdateOperators(
+                        parent.updateDocument,
+                        "\"\$unset\": { $unsetFields }",
+                    ),


UnsetClause returns a new Update without propagating parent.namedParameters. This drops any parameters accumulated earlier in the update chain (and makes it hard to add parameterized values later). Include namedParameters = parent.namedParameters (and merge additional parameters if introduced).

Suggested change

),

),

namedParameters = parent.namedParameters,

Copilot · 2026-03-25T15:55:36Z

+                val parent = clause.parentClause.toDopeQuery(this) as MongoDopeQuery.Aggregation
+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages,
+                    bucket = clause.fromable as? Bucket,


bucket = clause.fromable as? Bucket will set bucket to null for valid Fromable implementations that are not a Bucket (e.g. UseKeysClass, UseIndex in core). This leads to NPEs later (query.bucket!!) and silently produces unusable aggregations. Consider either (1) resolving/unwraping these Fromable wrappers to the underlying Bucket, or (2) failing fast with a clear error when fromable isn't supported for Mongo.

Suggested change

bucket = clause.fromable as? Bucket,

bucket = (clause.fromable as? Bucket)

?: error("Mongo FromClause requires a Bucket fromable, but got ${clause.fromable::class.simpleName}"),

Copilot · 2026-03-25T15:55:36Z

+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages + lookupStages,


For JOINs, $lookup stages are appended to parent.stages, but parent.stages already contains the $project produced by SelectClause. This means projection happens before the join, potentially removing join key fields and changing semantics (SQL join is evaluated before SELECT/projection). Consider restructuring stage assembly so $project is applied after $lookup/$match/$sort/$limit (or inserting $lookup before the first $project).

Suggested change

MongoDopeQuery.Aggregation(

stages = parent.stages + lookupStages,

// Insert lookup stages before the first $project (SELECT projection) so joins

// are evaluated before projection, preserving join keys.

val firstProjectIndex = parent.stages.indexOfFirst { it.contains("\$project") }

val newStages =

if (firstProjectIndex == -1) {

parent.stages + lookupStages

} else {

parent.stages.take(firstProjectIndex) + lookupStages + parent.stages.drop(firstProjectIndex)

}

MongoDopeQuery.Aggregation(

stages = newStages,

Copilot · 2026-03-25T15:55:38Z

    testImplementation(kotlin("test"))
    testImplementation("com.couchbase.client:kotlin-client:1.5.0")
    testImplementation("org.testcontainers:couchbase:1.21.4")
+    testImplementation("io.github.classgraph:classgraph:4.8.174")


io.github.classgraph:classgraph was added as a test dependency, but there are currently no references to ClassGraph in the couchbase module sources/tests. If it's not needed for upcoming work, consider removing it to avoid extra dependency/download time and keep the test classpath smaller.

Suggested change

testImplementation("io.github.classgraph:classgraph:4.8.174")

Copilot · 2026-03-25T15:55:38Z

+            is FromClause<*> -> {
+                val parent = clause.parentClause.toDopeQuery(this) as MongoDopeQuery.Aggregation
+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages,


In FromClause resolution, namedParameters from the parent aggregation are dropped (the new Aggregation uses the default empty parameters). This will lose any parameters produced in the SELECT expression(s) and can break queries once parameter expressions are supported. Propagate namedParameters = parent.namedParameters when returning the Aggregation.

Suggested change

stages = parent.stages,

stages = parent.stages,

namedParameters = parent.namedParameters,

Copilot · 2026-03-25T15:55:38Z

+                val mergeable = clause.mergeable as Bucket
+                val condition = clause.condition ?: error("JOIN ON requires a condition for Mongo lookup")
+                val asName = clause.bucket?.name ?: mergeable.name
+
+                val lookup = trySimpleLookup(mergeable.name, asName, condition)
+                    ?: buildPipelineLookup(mergeable.name, asName, condition)


val mergeable = clause.mergeable as Bucket is an unsafe cast: MergeableClause.mergeable is a Joinable, and core provides Joinable implementations that are not Bucket (e.g. UseKeysClass, UseIndex). This will throw ClassCastException at runtime for those join forms. Consider supporting these wrappers (unwrap to their bucket) or rejecting them explicitly with a clear error message.

Suggested change

val mergeable = clause.mergeable as Bucket

val condition = clause.condition ?: error("JOIN ON requires a condition for Mongo lookup")

val asName = clause.bucket?.name ?: mergeable.name

val lookup = trySimpleLookup(mergeable.name, asName, condition)

?: buildPipelineLookup(mergeable.name, asName, condition)

val mergeableBucket = when (val mergeable = clause.mergeable) {

is Bucket -> mergeable

else -> error(

"Mongo merge (JOIN) currently supports only Bucket-based mergeables, " +

"but was: ${mergeable::class.qualifiedName ?: mergeable::class.simpleName}"

)

}

val condition = clause.condition ?: error("JOIN ON requires a condition for Mongo lookup")

val asName = clause.bucket?.name ?: mergeableBucket.name

val lookup = trySimpleLookup(mergeableBucket.name, asName, condition)

?: buildPipelineLookup(mergeableBucket.name, asName, condition)

Copilot · 2026-03-25T15:55:39Z

+                val escaped = expression.value.replace("\\", "\\\\")
+                MongoDopeQuery.ExpressionFragment(queryString = "\"$escaped\"")
+            }
+
+            else -> TODO("not yet implemented: $expression")
+        }


StringPrimitive escaping currently only replaces backslashes. If the string contains quotes, control characters (\n, \r, \t), or other JSON-special characters, the produced query will be invalid JSON and could allow query-shape injection. Use a proper JSON string escaper (or emit primitives via BSON/JSON utilities) so all required characters are escaped safely.

Suggested change

val escaped = expression.value.replace("\\", "\\\\")

MongoDopeQuery.ExpressionFragment(queryString = "\"$escaped\"")

}

else -> TODO("not yet implemented: $expression")

}

val escaped = escapeJsonString(expression.value)

MongoDopeQuery.ExpressionFragment(queryString = "\"$escaped\"")

}

else -> TODO("not yet implemented: $expression")

}

private fun escapeJsonString(value: String): String {

val sb = StringBuilder(value.length + 16)

for (ch in value) {

when (ch) {

'\"' -> sb.append("\\\"")

'\\' -> sb.append("\\\\")

'\b' -> sb.append("\\b")

'\u000C' -> sb.append("\\f")

'\n' -> sb.append("\\n")

'\r' -> sb.append("\\r")

'\t' -> sb.append("\\t")

else -> {

if (ch < ' ') {

val code = ch.code

sb.append("\\u")

sb.append(((code shr 12) and 0xF).toString(16))

sb.append(((code shr 8) and 0xF).toString(16))

sb.append(((code shr 4) and 0xF).toString(16))

sb.append((code and 0xF).toString(16))

} else {

sb.append(ch)

}

}

}

}

return sb.toString()

}

Copilot · 2026-03-25T15:55:39Z

+
+                MongoDopeQuery.Aggregation(
+                    stages = parent.stages + lookupStages,
+                    bucket = parent.bucket,


MergeableClause resolution creates a new Aggregation but doesn't propagate parent.namedParameters, so any parameters accumulated before the JOIN are lost. Even if Mongo currently inlines primitives, this will break as soon as parameter expressions are supported. Pass through namedParameters = parent.namedParameters (and merge with any lookup-related parameters if added later).

Suggested change

bucket = parent.bucket,

bucket = parent.bucket,

namedParameters = parent.namedParameters,

…dope-293-mongo-poc

…requirements in clauses

- Implement the MongoDB resolver: expressions, aggregates, GROUP BY and clauses, with integration tests against a real MongoDB. - Move N1QL-only functions and query parameters from core into the couchbase module - Split crystal-map-connector into a core-based base plus crystal-map-connector-couchbase so Crystal Map works with both backends.

Resolve conflicts from main's DOPE-281 token/bucket refactor: adopt main's reorganized token functions (function.token package) and keep the string functions in core, reverting the couchbase move for the string category only. Non-string module separation (query parameters, WITHIN, and the array/date/object N1QL functions) is preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

jansigi added 5 commits March 20, 2026 10:28

DOPE-293: mongo poc with aggregate and find

66a6f16

DOPE-293: implement MongoDB integration with query resolvers and tests

1828ffd

DOPE-293: use stages instead of queryString in MongoDopeQuery.kt

67cc31b

DOPE-293: enhance ClauseResolver with delete and update clause handling

1028b7b

DOPE-293: enhance MongoDopeQuery with update and delete operations

7801d96

jansigi requested review from Copilot and pgruntz March 25, 2026 15:46

jansigi self-assigned this Mar 25, 2026

Copilot started reviewing on behalf of jansigi March 25, 2026 15:47 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

jansigi and others added 6 commits March 30, 2026 10:36

Merge branch 'main' into feature/mongo-poc

8f41e8f

DOPE-293: refactor ClauseResolver and MongoDopeQuery to use parameters

4ba5996

Merge remote-tracking branch 'origin/feature/mongo-poc' into feature/…

84be60c

…dope-293-mongo-poc

DOPE-293: enhance MongoDopeQuery with error handling for Bucket type …

62d5b8d

…requirements in clauses

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Feature/mongo poc#94

Feature/mongo poc#94
jansigi wants to merge 11 commits into
mainfrom
feature/mongo-poc

jansigi commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	stages = parent.stages + "{ \$unwind: \"\$${clause.arrayTypeField.name}\" }",
	stages = parent.stages + "{ \$unwind: \"\$${clause.arrayTypeField.name}\" }",
	namedParameters = parent.namedParameters,

	bucket = clause.fromable as? Bucket,
	bucket = (clause.fromable as? Bucket)
	?: error("Mongo FromClause requires a Bucket fromable, but got ${clause.fromable::class.simpleName}"),

		MongoDopeQuery.Aggregation(
		stages = parent.stages + lookupStages,

-                MongoDopeQuery.Aggregation(
-                    stages = parent.stages + lookupStages,
+                // Insert lookup stages before the first $project (SELECT projection) so joins
+                // are evaluated before projection, preserving join keys.
+                val firstProjectIndex = parent.stages.indexOfFirst { it.contains("\$project") }
+                val newStages =
+                    if (firstProjectIndex == -1) {
+                        parent.stages + lookupStages
+                    } else {
+                        parent.stages.take(firstProjectIndex) + lookupStages + parent.stages.drop(firstProjectIndex)
+                    }
+                MongoDopeQuery.Aggregation(
+                    stages = newStages,

	stages = parent.stages,
	stages = parent.stages,
	namedParameters = parent.namedParameters,

-                val mergeable = clause.mergeable as Bucket
-                val condition = clause.condition ?: error("JOIN ON requires a condition for Mongo lookup")
-                val asName = clause.bucket?.name ?: mergeable.name
-                val lookup = trySimpleLookup(mergeable.name, asName, condition)
-                    ?: buildPipelineLookup(mergeable.name, asName, condition)
+                val mergeableBucket = when (val mergeable = clause.mergeable) {
+                    is Bucket -> mergeable
+                    else -> error(
+                        "Mongo merge (JOIN) currently supports only Bucket-based mergeables, " +
+                            "but was: ${mergeable::class.qualifiedName ?: mergeable::class.simpleName}"
+                    )
+                }
+                val condition = clause.condition ?: error("JOIN ON requires a condition for Mongo lookup")
+                val asName = clause.bucket?.name ?: mergeableBucket.name
+                val lookup = trySimpleLookup(mergeableBucket.name, asName, condition)
+                    ?: buildPipelineLookup(mergeableBucket.name, asName, condition)

-                val escaped = expression.value.replace("\\", "\\\\")
-                MongoDopeQuery.ExpressionFragment(queryString = "\"$escaped\"")
-            }
-            else -> TODO("not yet implemented: $expression")
-        }
+                val escaped = escapeJsonString(expression.value)
+                MongoDopeQuery.ExpressionFragment(queryString = "\"$escaped\"")
+            }
+            else -> TODO("not yet implemented: $expression")
+        }
+    private fun escapeJsonString(value: String): String {
+        val sb = StringBuilder(value.length + 16)
+        for (ch in value) {
+            when (ch) {
+                '\"' -> sb.append("\\\"")
+                '\\' -> sb.append("\\\\")
+                '\b' -> sb.append("\\b")
+                '\u000C' -> sb.append("\\f")
+                '\n' -> sb.append("\\n")
+                '\r' -> sb.append("\\r")
+                '\t' -> sb.append("\\t")
+                else -> {
+                    if (ch < ' ') {
+                        val code = ch.code
+                        sb.append("\\u")
+                        sb.append(((code shr 12) and 0xF).toString(16))
+                        sb.append(((code shr 8) and 0xF).toString(16))
+                        sb.append(((code shr 4) and 0xF).toString(16))
+                        sb.append((code and 0xF).toString(16))
+                    } else {
+                        sb.append(ch)
+                    }
+                }
+            }
+        }
+        return sb.toString()
+    }

	bucket = parent.bucket,
	bucket = parent.bucket,
	namedParameters = parent.namedParameters,

Uh oh!

Conversation

jansigi commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants