Morphium 6.3.0 — What's New?

Morphium 6.3.0 is the largest release in the 6.x line so far. Its centre of gravity is not the client this time: it is PoppyDB and the in-memory driver underneath it. Both went from "good enough to run your tests against" to something you can actually operate — authentication, configuration files, memory guard rails, real size limits, and a replication path that no longer loses or reorders anything. On top of that, two new optional modules bring Morphium into Jakarta Data and Quarkus applications, and messaging gained a third implementation.

If you only use Morphium against real MongoDB and never touch the embedded driver or PoppyDB, most of this release passes you by — jump to Messaging and Upgrading.

Two New Optional Modules

morphium-jakarta-data

A Jakarta Data 1.0 provider on top of Morphium's existing query engine. You declare a repository interface, Morphium derives the queries:

Method-name derivation with the full standard keyword set, JDQL via @Query (including GROUP BY/HAVING, compiled into a Morphium aggregation pipeline), @Find/@Delete with explicit @By binding, offset pagination (Page<T>) and cursor/keyset pagination (CursoredPage<T>), static and dynamic sorting.

The module is deliberately framework-agnostic — plain Java, no Quarkus, no Spring, no DI container — because it is meant to be consumed by framework integrations rather than added directly. The dependency direction is strictly one-way: core knows nothing about Jakarta Data, so declaring only de.caluga:morphium does not put jakarta.data-api on your classpath.

quarkus-morphium

The Quarkus integration, built on the module above: a CDI producer for Morphium, type-safe configuration via @ConfigMapping (quarkus.morphium.*), declarative @MorphiumTransactional transactions with CDI events, MicroProfile health checks, Dev Services (an automatically started MongoDB container, optionally as a single-node replica set), a Dev UI card, and GraalVM native-image support with automatic reflection registration for every @Entity. Repository implementations are generated at build time via Gizmo bytecode — no runtime reflection, no dynamic proxies.

If you already use this extension, its Maven coordinates changed. It previously published as io.quarkiverse.morphium:quarkus-morphium:1.2.0, but it does not actually live in the Quarkiverse organization, so it now follows Morphium's own groupId and versions in lockstep with the reactor:

No package renames, no API changes — only the coordinates move.

Both modules originate from Heiko Bardioc's morphium-jakarta-data and quarkus-morphium repositories, which are being archived now that their content lives in the main repository. Building the reactor with -DskipExtensions still produces a core-only build.

Messaging

DualChannelMessaging — a third implementation, in beta

Load measurements showed that request/reply throughput on MongoDB is delivery-bound rather than write-bound: a single change-stream cursor hands out majority-committed events at a fixed cadence, which caps sustained throughput regardless of how fast you offer messages. MultiCollectionMessaging did better in those runs — but not because of its per-topic collection split (on mongod every cursor tails the whole oplog anyway). The effective mechanism was its second cursor, for answers and direct messages.

DualChannelMessaging ports exactly that one mechanism onto the standard layout: the same single collection and cursor for broadcast and topic traffic, plus a dedicated per-recipient collection with its own cursor and dispatcher thread for directed messages and answers.

One thing to be clear about before you switch anything: every participant on a queue has to run the same messaging implementation. That was already true between SingleCollectionMessaging and MultiCollectionMessaging — their collection layouts have nothing in common — and Dual Channel is no exception. There is no dual-read/dual-write bridge, and a mismatch does not fail loudly: broadcast and topic traffic keeps flowing (Dual Channel's main lane is byte-for-byte Standard's), but a Standard node waiting for an answer from a Dual Channel responder waits forever, because that answer was written to the requester's DM collection, which Standard never reads. So this is a big-bang switch: stop the consumers, change every node's configuration, restart — and drain or pause request/reply traffic while you do it. Every DualChannelMessaging instance logs a WARN on startup to remind you.

It is marked beta on purpose: the measured benefit turned out smaller and more nuanced than the original motivation suggested. Past saturation it trades a little throughput for markedly better tail latency — p99 of 519 ms versus 723 ms for Standard and 2044 ms for MultiCollection in the steady-state window. Opt-in while it gathers real-world mileage; see the messaging implementations comparison.

And in every implementation

• One database roundtrip less per message — non-exclusive messages are now processed straight from the change-stream fullDocument instead of being re-read.
• Requeued messages are delivered event-driven instead of waiting for the next poll.
• Configurable default TTL and fallback-poll cadence, and the fallback poll is now driven by change-stream liveness rather than a fixed timer.
• A processing decision trace for diagnosing answer timeouts — it tells you why a message was or was not processed.
• Skipped messages were wrongly marked "recently completed", which blocked requeues for 10 seconds. Fixed.

PoppyDB Becomes Operable

The server previously had exactly one deployment mode: wide open. That is fine for tests and nothing else. 6.3.0 closes the gap.

Authentication. Server-side SCRAM-SHA-1 and SCRAM-SHA-256 (RFC 5802/7677, validated against the RFC test vectors), including MongoDB's specifics — MD5-digested password for SHA-1, SASLprep for SHA-256, the three-step exchange that clients like mongosh use. createUser really creates users now, stored mongod-shaped in admin.system.users. Verification is always active; enforcement is opt-in:

With --auth, a connection may only run the handshake, SASL, logout, ping and buildInfo until it completes a SCRAM exchange; everything else is refused with code 13. There is no localhost exception. Without --auth, nothing changes for existing setups. Wrong passwords and unknown users are rejected indistinguishably, so there is no user enumeration. Authorization is authentication-only for now — roles are stored but not evaluated.

Users you can put in version control. --rootUser only ever provisioned a single admin, so any real user set still had to be created by hand. --users-file provisions declaratively, with upsert semantics and a version gate so a config-management run can apply it repeatedly without surprises. And users now replicate across the replica set, so they survive a failover instead of existing only on whichever node happened to create them.

Configuration files. --cfg <file> (or -f) keeps secrets off the command line and out of the process table; --no-config ignores them entirely. --print-config shows the effective configuration, --check-config validates without starting. And --log-level exists, so the server no longer logs everything at DEBUG.

A DevOps command surface. Live currentOp/killOp, rs.conf(), listCommands, hostInfo, and real connection gauges. dbHash computes a per-collection hash in a canonical document order, so two replica-set members holding the same data produce the same hash even though sync materialized their collections in different order — a one-command consistency check, deliberately answered on secondaries too. validate is a real check rather than a stub: it walks every index and reports entries pointing at documents that no longer exist and documents missing from an index. top now fails with an explicit CommandNotSupported instead of a generic "command not found", because real mongod has the command and the error should say why PoppyDB does not.

Guard Rails

An in-memory store dies of OOM when producers outrun consumers — and a replica set dies completely, because replication copies the data volume to every node. Two watermarks now guard the write path:

Crossing the warn threshold logs once. Above the reject threshold, document-creating writes are refused with a mongod-shaped ExceededMemoryLimit (code 146) that clients should treat as retryable backpressure. Updates, deletes and TTL expiry stay allowed — the drain paths have to keep working, or the system could never get back under the watermark. Replication and initial sync bypass the guard, because a secondary refusing what the primary accepted would silently diverge.

Both stages decide on the post-GC live set, not on raw heap occupancy. That detail is the whole feature: with -Xms equal to -Xmx, the raw used/max gauge routinely reads above 90% under allocation-heavy load even when the next GC would free most of it. The first overnight replica-set CI run proved it — the raw-gauge version rejected the writes of eight otherwise-green messaging test classes on a heap that GC promptly dropped to 46%.

Two related limits are now honest as well. The 16MB BSON document limit was previously only advertised (and by the embedded driver as a fantasy 128MB), never enforced — updates could grow documents without any bound, which no real MongoDB would accept. It is now enforced exactly like mongod, down to the 16KB internal margin on update results, and rolled back atomically on violation. And maxMessageSizeBytes is respected end-to-end: batching used to be count-based only, so 1000 × 1MB documents went out as one ~1GB message that any real MongoDB answers by closing the connection. Write commands now split oversized payloads like the official drivers do, folding the results back into one mongod-shaped answer with the write-error indices mapped to your original statement positions.

The In-Memory Driver Closes the Gap to mongod

This is the least glamorous part of the release and probably the most valuable: a long correctness push, driven by issue after issue, so that tests passing against the in-memory driver actually mean something.

New aggregation capability: $merge, $documents, $densify, $fill, $setWindowFields (with its full window-function set), $collStats, $listSessions — and a real $out instead of a pretend one. Roughly 40 expression operators were implemented, and three that had been silently mis-calculating were fixed. The Aggregator gained typed builder methods for the new stages.

Updates: the positional operators $, $[] and $[<identifier>] with arrayFilters (also exposed on the Query API), plus $bit.

And a long list of things that were quietly wrong, of which the most alarming is that $geoWithin with $center, $centerSphere or $polygon matched every document in the collection. Date operators now use UTC with a 1-based $month and real ISO week fields; $dateFromParts returns a date instead of its own JSON shape; $project inclusion mode actually restricts the output to the selected fields; $sample larger than the collection returns everything instead of throwing; $indexStats no longer silently runs $geoNear; aggregation stages that used to silently run $bucket now error. Unknown commands are answered the way mongod answers them rather than throwing.

Replication and Failover

The failure modes fixed here are the kind you only find by running the thing:

• A re-syncing secondary broadcast its own initial-sync wipe as change-stream drop events — which let stale watchers destroy admin.system.users cluster-wide during a stepdown.
• Replication is now lossless and order-preserving, and covers index definitions, not just documents.
• A demoted leader could keep primary == true forever after a rapid leadership flap.
• rs.status() reported a peer that died along with the failover as SECONDARY forever, and it spoke Raft terminology while mis-identifying wildcard-bound nodes.
• --auth/--ssl now actually work on a replica set — the internal election and replication channel was always plaintext and unauthenticated.
• On the client side, the failover read path could throw a raw NPE past every retry, and getLastConnectFailure() stayed stale after recovery.
• startPoppyDB.sh's "port already in use, skipping node" did not, in fact, skip.

Manual failover reproduction has been replaced by an automated test that rewrites the wire protocol through a proxy, so these stay fixed.

Performance

• Insert's duplicate-_id pre-check is an O(1) index lookup instead of a full collection scan under the write lock. That scan was the dominant per-insert cost for the messaging workload.
• The change-stream before-image is no longer deep-copied twice per watched update.
• A rebuild ping-pong between an open transaction and concurrent readers on the same collection was eliminated: the index store is now handed to the new owner atomically rather than evicted, turning 20 rebuilds into 2 in the measured scenario.
• The change-stream replay buffer is bounded in O(1), and dbStats/collStats report real sizes instead of zeros.

The Bug Worth Reading Twice

If you use Morphium messaging on PoppyDB, this one is why you want 6.3.0:

invalidateTtlQueue() discards a collection's expiry queue at every structural change and relies on a lazy rebuild on the next miss. Only the sweep honoured that contract — ttlEnqueue() installed a fresh queue containing nothing but the document it was called for. That queue was then no longer absent, so the rebuild never fired again, and every document that existed before the invalidation permanently lost its expiry tracking.

Msg.deleteAt is TTL-indexed, so this is exactly how Morphium messaging cleans up after itself. A messaging node starting against a PoppyDB that already held messages opened the window, and from that point on the pre-existing messages never expired: unbounded growth of the message collection.

Transactions had a similar class of problem — a CollectionIndexStore built before a transaction started stayed stale for its whole lifetime and could silently lose an update on commit, and aborted or committed transactions could leave stale entries behind that produced false duplicate-key errors on a provably empty collection. All fixed.

Upgrading

And if you use PoppyDB for testing:

No dependency version bumps in this release — Netty, BSON, SLF4J and Logback are unchanged from 6.2.10.

For the breaking changes, the deprecations and the quarkus-morphium groupId move, the migration guide walks through them in order. Full release notes are in the CHANGELOG on GitHub.

Thanks in particular to Heiko Bardioc, whose downstream applications found several of the transaction and index-store regressions in this cycle before they could reach a release.