Long story short, I had a server running Ubuntu 20.04 and unsurprisingly it’s been there for more than 4 years. For no reason I decided to give it an upgrade, actually 2 upgrades, ie. 20.04 –> 22.04 –> 24.04. And I totally regretted.
The Ubuntu Linux system upgrade went well and I completely ignored the warning that MongoDB 6 is no longer supported in Ubuntu 24.04 and the lowest supported MongoDB version is 8, and it failed to start. I went to check the logs (by default /var/log/mongodb/mongod.log) and saw the following lines:
{"t":{"$date":"2024-11-10T04:19:58.278+00:00"},"s":"F", "c":"STORAGE", "id":28661, "ctx":"initandlisten","msg":"Unable to read the storage engine metadata file","attr":{"error":{"code":38,"codeName":"FileNotOpen","errmsg":"Failed to read metadata from /var/lib/mongodb/storage.bson"}}}
{"t":{"$date":"2024-11-10T04:19:58.278+00:00"},"s":"F", "c":"ASSERT", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28661,"file":"src/mongo/db/storage/storage_engine_metadata.cpp","line":102}}
{"t":{"$date":"2024-11-10T04:19:58.278+00:00"},"s":"F", "c":"ASSERT", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}Basically the fact is MongoDB 8 refused to read from MongoDB 6 database files. Normally one can re-install MongoDB 6, do a mongodump , then install a fresh MongoDB 8 and use mongorestore to restore all the databases in 8’s format. In my case, I could not go back to 6 because 6 is no longer supported in Ubuntu 24.04. But I can run a MongoDB 6 Docker container in whatever system that supports Docker! Here’s how I did it:
# I ran the following as root
# copy the DB files and work on the copy just in case
rsync -av /var/lib/mongodb ~/
# run the mongodb 6 docker image with the database directory mounted
docker pull mongo:6.0.19-jammy
docker run --rm -v `pwd`/mongodb:/data/db mongo:6.0.19-jammy
...
{"t":{"$date":"2024-11-10T04:43:10.051+00:00"},"s":"I", "c":"NETWORK", "id":23016, "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
# in the other shell, use `mongodump` to dump all data
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4ee1c9ffd25b docker.io/library/mongo:6.0.19-jammy mongod 21 seconds ago Up 21 seconds infallible_shockley
docker exec -ti infallible_shockley bash
root@4ee1c9ffd25b:/# cd /data/db
root@4ee1c9ffd25b:~# mkdir dump
root@4ee1c9ffd25b:~# cd dump/
# saving the dump at /data/db/dump, so it will be in the ~/mongodb/dump in host system
root@4ee1c9ffd25b:~/dump# mongodump
2024-11-10T04:46:40.061+0000 writing admin.system.users to dump/admin/system.users.bson
...
root@4ee1c9ffd25b:~/dump# exit
# in host system, the dump is at ~/mongodb/dump
cd ~/mongodb
# start the restoration
mongorestore dump/
...
2024-11-10T05:21:00.686+0000 309001 document(s) restored successfully. 0 document(s) failed to restore.Ta-da! 🙂