Loading Metadata into MarkLogic

To support the University of Chicago Library's OAI-PMH provider, metadata need to be loaded into MarkLogic as Europeana Data Model (EDM) triples. Because EDM is, in a manner of speaking, an application profile of RDF, and since the canonical expression of RDF is XML, we may express EDM/RDF as XML "records" for the sake of efficiency: it is far more performant to retrieve EDM/RDF as XML from MarkLogic than to assemble what are effectively metadata records from RDF triples. Here is an example of an EDM/RDF record expressed as XML:

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:bf="http://id.loc.gov/ontologies/bibframe/"
   xmlns:dc="http://purl.org/dc/elements/1.1/"
   xmlns:dcterms="http://purl.org/dc/terms/"
   xmlns:dma="http://lib.uchicago.edu/dma/"
   xmlns:edm="http://www.europeana.eu/schemas/edm/"
   xmlns:erc="http://purl.org/kernel/elements/1.1/"
   xmlns:fn="http://www.w3.org/2005/xpath-functions"
   xmlns:icu="http://lib.uchicago.edu/icu/"
   xmlns:lexvo="https://www.iso.org/standard/39534.html"
   xmlns:olac="http://www.language−archives.org/OLAC/metadata.html"
   xmlns:ore="http://www.openarchives.org/ore/terms/"
   xmlns:premis="https://id.loc.gov/ontologies/premis−3−0−0.html"
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
   xmlns:rel="http://id.loc.gov/vocabulary/relators/"
   xmlns:uchicago="http://lib.uchicago.edu/"
   xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
   xml:base="mlc.ttl">
  <edm:ProvidedCHO rdf:about="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85">
    <uchicago:identifier>9580</uchicago:identifier>
    <dc:date>2018</dc:date>
    <dc:description>Contents=Track 1 of the tape (=Discs 96A - 102B) (There is no Track 2.)</dc:description>
    <dc:identifier>https://n2t.net/ark:61001/b2bt7k86vq85</dc:identifier>
    <dc:title>yua-and-txts-7</dc:title>
    <dcterms:alternative>yua-and-txts-7</dcterms:alternative>
    <dcterms:isFormatOf rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2593395ct93"/>
    <dcterms:isPartOf rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2198gp3vb0h"/>
    <dcterms:medium>Sound</dcterms:medium>
    <dcterms:spatial>(:unav)</dcterms:spatial>
    <dcterms:temporal>2018-04-04</dcterms:temporal>
    <erc:what>yua-and-txts-7</erc:what>
    <erc:when>2018</erc:when>
    <erc:where>https://n2t.net/ark:61001/b2bt7k86vq85</erc:where>
    <erc:who>(:unav)</erc:who>
  </edm:ProvidedCHO>
  <ore:Aggregation rdf:about="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/aggregation">
    <edm:aggregatedCHO rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85"/>
    <edm:dataProvider>University of Chicago Library</edm:dataProvider>
    <edm:isShownAt>https://n2t.net/ark:61001/b2bt7k86vq85</edm:isShownAt>
    <edm:isShownBy rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b22n3552x113"/>
    <edm:object rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b22n3552x113"/>
    <edm:provider>University of Chicago Library</edm:provider>
    <edm:rights rdf:resource="https://rightsstatements.org/vocab/NoC-US/1.0/"/>
    <ore:isDescribedBy rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/rem"/>
    <fn:collection rdf:resource="http://lib.uchicago.edu/mlc"/>
    <fn:collection>dma</fn:collection>
  </ore:Aggregation>
  <ore:Proxy rdf:about="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/file.dc.xml">
    <dc:format>application/xml</dc:format>
    <ore:proxyFor rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85"/>
    <ore:proxyIn rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/aggregation"/>
  </ore:Proxy>
  <edm:WebResource rdf:about="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/file.wav">
    <dc:format>audio/x-wav</dc:format>
    <dc:identifier>https://n2t.net/ark:61001/b2bt7k86vq85/file.wav</dc:identifier>
    <dc:identifier>https://uchicago.hosted.panopto.com/Panopto/Pages/Embed.aspx?id=5e5658ee-1600-46b9-baa6-aee40065ee53</dc:identifier>
    <premis:fixity>
      <ns662:sha512 xmlns:ns662="https://id.loc.gov/vocabulary/preservation/cryptographicHashFunctions/">
        <rdf:value>2b3e3fc2a8e0246f5ff635554d9db4795840dddbd4a14a1700ebb4a6ed3bdfdcb6bcef7e91b9f0d98e71693731197d169ddb011c7e1cc11ffe398deec985587b</rdf:value>
      </ns662:sha512>
    </premis:fixity>
    <premis:originalName>yua-and-txts-7.wav</premis:originalName>
    <premis:size rdf:datatype="http://www.w3.org/2001/XMLSchema#integer">767689928</premis:size>
  </edm:WebResource>
  <ore:ResourceMap rdf:about="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/rem">
    <dc:creator rdf:resource="https://dldc.lib.uchicago.edu/"/>
    <dcterms:created rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-10-09T13:17:56.335405-05:00</dcterms:created>
    <dcterms:modified rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2023-10-09T13:17:56.335405-05:00</dcterms:modified>
    <ore:describes rdf:resource="https://ark.lib.uchicago.edu/ark:61001/b2bt7k86vq85/aggregation"/>
  </ore:ResourceMap>
</rdf:RDF>

If the OAI-PMH provider is asked to return results as unqualified Dublin Core, which is a requirement of the OAI-PMH protocol, it will do so from the preceding as follows:

        for $i in fn:collection($set) 
        return
        <record>
        <header>
        <identifier>{fn:data($i//edm:ProvidedCHO/dc:identifier)}</identifier>
        <datestamp>{fn:data($i//ore:ResourceMap/dcterms:modified)}</datestamp>
        <setSpec>{$set}</setSpec>
        </header>
        <metadata>
        <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://ww
w.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.x
sd">, for example:
        {
        $i//edm:ProvidedCHO/dc:contributor,
        $i//edm:ProvidedCHO/dc:coverage,
        $i//edm:ProvidedCHO/dc:creator,
        $i//edm:ProvidedCHO/dc:date,
        $i//edm:ProvidedCHO/dc:description,
        $i//edm:ProvidedCHO/dc:format,
        $i//edm:ProvidedCHO/dc:identifier,
        $i//edm:ProvidedCHO/dc:language,
        $i//edm:ProvidedCHO/dc:publisher,
        $i//edm:ProvidedCHO/dc:relation,
        $i//edm:ProvidedCHO/dc:rights,
        $i//edm:ProvidedCHO/dc:source,
        $i//edm:ProvidedCHO/dc:subject,
        $i//edm:ProvidedCHO/dc:title,
        $i//edm:ProvidedCHO/dc:type
        }
        </oai_dc:dc>
        </metadata>
        </record>

If it is asked to return results as EDM, a requirement for UChicagoNode, it will do so as follows:

for $i in fn:collection($set) 
return
<record>
<header>
<identifier>{fn:data($i//edm:ProvidedCHO/dc:identifier)}</identifier>
<datestamp>{fn:data($i//ore:ResourceMap/dcterms:modified)}</datestamp>
<setSpec>{$set}</setSpec>
</header>
<metadata>
{$i}
</metadata>
</record>

Sets in OAI-PMH will be identified by the value of the uchicago:inCollection predicate in the ore:Aggregation. Values may include the following:

uchicago:inCollection <http://lib.uchicago.edu/uchicagonode>
uchicago:inCollection <http://lib.uchicago.edu/mlc>
uchicago:inCollection <http://lib.uchicago.edu/cailla> OR <http://lib.uchicago.edu/mcmca> OR <http://lib.uchicago.edu/ola>

We also need to support SPARQL queries against the same data, so we load them also as RDF, either as RDF/XML or as Turtle. While the preceding supports GetRecord and ListRecords, the following supports ListIdentifiers:

declare function query($graph) {
let $query := fn:string-join(('
prefix dc: <http://purl.org/dc/elements/1.1/>
prefix dcterms: <http://purl.org/dc/terms/>
prefix edm: <http://www.europeana.eu/schemas/edm/>
prefix ore: <http://www.openarchives.org/ore/terms/> 

select ?identifier ?datestamp',
$graph, '
where {
           { ?resource a edm:ProvidedCHO }
           { ?resource dc:identifier ?identifier }

           { ?aggregation edm:aggregatedCHO ?resource }

           { ?resourceMap ore:describes ?aggregation}
           { ?resourceMap dcterms:modified ?datestamp}

}', ''))
let $map := map:new((map:entry("graph", $graph)))
return sem:sparql($query, $map)
};

So the triples support OAI-PMH requests as well. (A SPARQL graph is equivalent to an XQuery fn:collection and the OAI-PMH set.)