Search for Hector Garcia-Molina and Jeff Ullman and Jennifer Widom.

A nit pick... You want to know "how to write an actual DBMS", not "how
to write an actual database".   In fact a DBMS writes an actual
database - not a human!  You need to ask for the best books on RDBMS
implementation, not RDB implementation.

I'm interested in this subject myself.  It's a huge subject area -
particularly when one gets into query optimisation.

I've found Jim Gray's work worth a read.

"Understanding MySQL internals" (published by O'Reilly) is not a bad
place to start. J.

Implementation books were rare twenty years ago so I'm not surprised
they apparently still are.  (The few I vaguely remember were also not
read by many people and went out of print quickly.)  It might help if
you stated your purpose, eg., just for illumination or because you want
to write something.  (If the latter, reading the little that's generally
available might just lead to re-inventing some pretty mediocre wheel or
other whereas working up your requirements from various points of view
such as language features, concepts such as whether you intend to define
keys up front or just cop out with a keyword, concurrency, recovery
strategy and so forth might then point you to more specific material in
the many papers that can be found with a bit of search work.)

These books I found useful for me

Transaction Processing: Concepts and Techniques by Jim Gray, Andreas
Reuter, Andreas Reuter
Database Systems : The Complete Book by Hector Garcia-Molina, Jennifer
Widom, Jeffrey D. Ullman
Database Management Systems by Raghu Ramakrishnan, Johannes Gehrke,
Johannes Gehrke