IBM's developerWorks has published an article I wrote about RELAX NG with custom datatype libraries. This article explores one of the most powerful but little-known features of RELAX NG: the ability to define new simple data types using Java code. This enables one to check constraints like a number is prime, every left parenthesis is matched by a properly balanced right parenthesis, or the value of an SKU attribute matches the value of an SKU field in an external database. None of these constraints are expressible in the W3C XML Schema Language.
Ispras Modis has posted Sedna 0.3, an open source native XML database for Windows written in C++ and Scheme and published under the Apache License 2.0. This is not currently recommended for production. Sedna has partial support for XQuery and its own declarative update language.