published : 2022-11-15 changed: 2022-11-15
category: Computer --> programming --> MongoDB --> morphium
Morphium originated in 2017 and has had 4 major releases since then. Now it's time to update some more and tweak some internals. This update is a major one, so parts of the internals have been completly rewritten while keeping Morphium itself sourcecode compatible.
The aim is to cut off old "braids" and update the code. It should remain as compatible as possible so that migration is easy.
Furthermore, the complexity should be reduced, there are some features in
Morphium that are certainly rarely used or not at all, such as "optimistic locking" with the use of automatic versioning. These "features" are removed and the code is streamlined.
Morphium should remain compatible with the older versions. That means the changes to
Morphium itself will be limited. The concepts will not change either, at most they will be tightened up a bit. So
Morphium is still the linchpin of the API and every write access will continue to run via
Morphium and every read access via
Query (exception: aggregation with the help of the aggregator). In short: the use of
Morphium essentially does not change.
The JDK used should also be updated during the refresh.
Morphium V4 stays on JDK1.8, from V5 we will support JDK11 and later. Of course, the code should also use the new features of JDK11.
In current tests, the
Morphium Object Mapper is still one of the fastest and most feature-rich object mappers that we have found. Compared to 'Jackson', the higher performance and the features for handling lists and maps (handling 'Generics') have to be mentioned in particular - this makes the
MorphiumObjectMapper really indispensable.
During the rewrite, however, some features that are also reflected in the object mapping were removed, which simplifies the code and makes it easier to maintain.
Morphium had declarative caching as part of the system from the start. This continues to be a key component. The cache implements the
JCache API and could therefore be used for other purposes as well. The other way around, you can also use a JCache in
Morphium for caching.
During the rewrite we will adapt the code to JDK11 and improve the structure of the code.
Another key component is messaging.
Particular attention will be paid to messaging to improve stability and reduce load. Specifically, by efficiently using the Driver API and WireProtocol. Another change will be the connection handling in messaging: currently messaging uses
Morphium itself as a connector to the mongo. Since none of the features of
Morphium are necessary for messaging or are sometimes even more of a hindrance, the messaging will communicate directly with the MongoDB in
V5. In the current V5 implementation, the messaging uses a dedicated connection to the MongoDB for the push notifications (
Watch) and a dedicated connection pool for the connection to the mongo.
This means that the messaging is a little more detached from
Morphium, generates less load and can be configured separately. This is implemented and the tests show, that the performance increases whereas the load on mongo is reduced.
This "thing" has also grown a little over the years and is currently hardly usable with the 100s of getters and setters. The settings have to be split up a bit and used in different categories. In particular, because
Morphium's own wire -protocol driver requires different settings than e.g. the
However, we only implement this for the Community Edition. This means that the enterprise features are not considered in the rewrite. It can also cause the drivers not to work properly with the Enterprise MongoDB version. The same currently applies to the Atlas MongoDB implementation, which will certainly be the first to be submitted. Support for the Enterprise Edition features may also be implemented in one of the following versions if required.
The rewrite is done from "below", i.e. first the Mongo driver (see below) was implemented, which implements the Wire protocol (currently only usable for MongoDB from V5! For older MongoDB versions please use
The necessities and the structure of this driver is mainly determined by the MongoDB and propagates the architecture "up". On the way there, unused features are removed (e.g. auto-versioning), others are improved simply by adapting them to the new drivers.
In the end, a
Morphium should come out that is almost 100% source compatible with the old version, but much more modern, slimmer, more beautiful!
Morphium is more or less a replacement for the existing mongoDB driver with a lot more features and in some places significantly more security and / or performance.
Currently (i.e. in
Morphium V4.x) there are 2 drivers:
MongoDriver in particular creates a bit of overhead because it is based on the official MongoDB driver, which unfortunately is now not just a driver, but also an object mapper. Many functions of the driver are not required at all, but some others are missing. So the aim is to introduce our own
WireProtocol[^1:WireProtocol is the protocol that MongoDB uses for communication] driver, which is optimized for the needs of
morphium. This can result in a significant performance advantage (currently between 10% and 20%).
2 drivers are implemented in V5:
SingleConnectDriver- a driver that only keeps a single connection to the MongoDb, but monitors it and also implements a failover if necessary. This is used, for example, in the new messaging substructure (see below)
PooledDriver- a full featured cluster connector with connection pooling and automatic failover.
Strictly speaking, there is a third driver, the
InMemoryDriver, which makes it possible to run
Morphium with an "in memory" mongo. This is particularly useful for tests. We use this as a TestDB in some projects and it is also comparable to other databases implemented for test purposes.
The InMemory database should be (nearly) feature compatible to a real MongoDB installation. We're not 100% there yet, but we've reached a good 90%. Most features are already working, but some things take a little longer and might come later (such as "real" text indexes).
This question is asked more often and I would like to say a few words about it:
The official MongoDB Java driver is not only a driver, but "embedded" in the driver is also an object mapper. This is probably super useful for many applications and completely sufficient.
Unfortunately not for us, because we want to configure some things rather than program them (declarative indices, etc.).
Two different ways of thinking collide. Which in the example of the MongoDB driver and
Morphium has repeatedly led to... discrepancies. Some changes to the driver API have been a lot of work in
MorphiumDriver project has been around for a few years. It was brought to life because we had problems with the official MongoDB driver, especially in the area of failover. Also, some features of the driver are at least a little questionable in my opinion.
These features, especially the object mapping, make access via
Morphium a bit unwieldy. And some things happen unnecessarily twice. And that's exactly why we want to try to develop our own driver that does exactly what we need - and nothing more.
When accessing the Mongo in the stack trace, there are currently a good 20 calls within the MongoDB driver. With the new driver that's a maximum of 5!
Morphium in version 5 got its own server, which is also based on the InMemory database. This is naturally intended exclusively for test purposes and does not offer any security or anything similar. However, it is always helpful for testing a (simple) in memory mongodb (with the limitations of the InMemoryDriver).
If you want to test it now:
MorphiumServer understands the following command line options:
--maxThreads: maximum number of threads in the server (= maximum number of connections)
--minThreads: minimal number of threads (kept ready)
--host: which host does this server report back as (in the response to the
--port: which port should be opened (all network IPs)
I would like to reiterate that this server is purely for testing purposes and is certainly not intended to be a replacement for any MongoDB installations.
Migration should be relatively easy since
Morphium's API itself hasn't changed. However, the configuration has changed:
ClassName, now the
DriverNameis specified. Valid drivers currently are:
MongoDriver. Unfortunately, the MongoDriver is currently not working and it is not 100% certain that it will remain part of
optimistic lockingfeature has been removed. Unfortunately, everyone who has used this feature currently has to find their own implementation
Currently, at the end of August 2022,
Morphium V5 is available as an 'ALPHA' version in Maven Central and can be used for your own tests. However, some features are not yet 100% implemented. This is how Morphium works only with MongoDB versions 5.0 and later. Also, the transaction feature is currently not working.
in the coming weeks these things will be addressed in this priority:
InMemoryDriver- add missing features, increase performance.
created Stephan Bösebeck (stephan)