Adam Retter

adam@evolvedbinary.com
 

XQuery / eXist-db Bootcamp for Humanists
CIHAM and HiSOMA
University of Lyon @ Online
2021-03-22

@adamretter

• Director and CTO of Evolved Binary

  • XQuery / XSLT / Schema / RelaxNG

  • Scala / Java / C++ / Rust*

  • Concurrency and Scalability

• Creator of FusionDB multi-model database (5 yrs.)

• Contributor to Facebook's RocksDB (5 yrs.)

• Core contributor to eXist-db XML Database (15 yrs.)

• Founder of EXQuery, and creator of RESTXQ

• Was a W3C XQuery WG Invited expert

• Me: www.adamretter.org.uk

• What are these, and how are they related?

  • FusionDB

    • Uses eXist-db components - 100% eXist-db API Compatible

    • Improved Storage, ACID Transactions, Instant Backup, etc.

    • Target - Larger projects and organisations

    • Source Available - Free For Dev. / Requires Licenses for Production Use

  • Elemental

    • Hard-fork of eXist-db - 100% eXist-db API Compatible (Release Q2 2021)

    • New Features and Fixes. Community Roadmap driven.

    • Target - Smaller projects and users

    • Source Available - Completely Free to Use

  • eXist-db

    • No one owns it

    • Open Source - Completely Free to Use

• What is REST?

• eXist-db's REST Server

• Using eXist-db's REST API

• How to design your own REST API

• Plain REST vs. controller.xq vs. RESTXQ

• REST - Representational State Transfer

  • ~ Architectural Styles and the Design of Network-based Software Architectures. Fielding, Roy. 2000.

  • A set of Principles, Properties, and Constraints

• Helps guide us to build well-designed Web Applications

• Focused on the use of HTTP

• RESTful Applications are applications that adhere to REST principles

  • Not unusual for applications to be partially RESTful

  • You likely use RESTful applications every day via your Web Browser

• Client / Server Architecture

  • Separate the UI concerns from the Storage concerns

  • The Web does this! i.e. Web Browser and Web Server

• Statelessness

  • Every request is isolated and can stand alone

• Cacheability

• Layered System

• Code on Demand

• Uniform Interface

  • URIs, HTTP Headers, and HyperMedia

• Refers to Extrinsic State (Application State)

• Specifically, NOT keeping Application State at the Server

  • Client maintains its own application state instead

  • Sends what is needed in each request

• Reduces load on the Server

  • No Session needed per-user

  • Reduces conditional computation

• Horizontal Scaling

  • Any server can process any request

• URI

  • One per resource (or collection of resources)

    • All apples (or the only apple): /apple

    • A specific apple: /apple/pink-lady

• Manipulation

  • Verbs = HTTP Methods: GET, HEAD, PUT, POST, PATCH*, DELETE

• Self-Descriptive Messages

  • Media Type

  • HTTP Headers: Content-Type, Accept

• Hypermedia (HATEOS)

  • Response should include links to further resources

  • e.g. HTML

• GET

  • Retrieve a (representation of a) Resource(s)

• HEAD

  • Get but with no representation (body), just metadata (headers)!

• PUT

  • Store/Replace a Resource at URI

• POST

  • Store a Resource as subordinate of URI

  • Update a Resource (consider PATCH instead)

• DELETE

  • Delete a Resource(s)

• Content Type

  • Indicates the Internet Media Type (Mime Type) of the Request/Response Body

  • e.g. Response from GET https://www.google.com:

    
    
    

  • Client POST/PUT/PATCH - tells the server the Media Type of the Resource it is sending

  • Server - tells the client the Media Type of the Resource it is returning

• Accept

  • Indicates one or more Internet Media Types that the client will accept for the Response Body

• Hyperlinks - Not just HTML!

• HTML

  • Unbiquitous - understood by your Web-browser

• JSON

• XML

  • Syntax for linking is up to you! Various standards exist

• API Base URI: /exist/rest/db

• REST API for Document Database CRUD Operations

  • Get Collection

  • Create Document

  • Get Document

  • Replace Document

  • Update Document

  • Delete Document

  • Delete Collection

• Additionally

  • Execute Dynamic Query

  • Execute Stored Query

• HTTP GET http://localhost:8080/exist/rest/db/

• Web Browser Example

• cURL Example:

• HTTP GET http://localhost:8080/exist/rest/db/

• Python Example:

• Result:

• HTTP PUT vs. POST

  • Subtly different

  • Think - Document vs. Collection of Documents

  • Can be confusing at first!

• HTTP PUT - to the URI of the Document

  • e.g. http://localhost:8080/exist/rest/db/apple/pink-lady.xml

  • NOTE: If document already exists, it will be replaced!

• HTTP POST - to the URI of the Collection in which to store the Document

  • e.g. http://localhost:8080/exist/rest/db/

  • Response will tell you the new URI of the Document

  • WARNING: eXist-db does NOT support this correctly!

    • eXist-db uses POST only for XQuery and XUpdate.

• HTTP PUT http://localhost:8080/exist/rest/db/apple/pink-lady.xml

• cURL Example

• TIP: Instead of specifying body inline with -d, use a file:

• Similar to Get Collection

• HTTP GET http://localhost:8080/exist/rest/db/apple/pink-lady.xml

• cURL Example

• Uses XUpdate to describe the changes to make

  • NOTE: eXist-db should probably use HTTP PATCH instead of POST!

• HTTP POST http://localhost:8080/exist/rest/db/apple/pink-lady.xml

• cURL Example

• Can also be applied to an entire Collection!

  • HTTP POST http://localhost:8080/exist/rest/db/

• Alternatives:

  • HTTP POST an XQuery containing XQuery Update expression(s)

  • HTTP GET or POST to a Stored XQuery

• Similar to Get Document

• HTTP DELETE http://localhost:8080/exist/rest/db/apple/pink-lady.xml

• cURL Example

• Similar to Get Collection

• HTTP DELETE http://localhost:8080/exist/rest/db/apple

• cURL Example

• You can either HTTP GET or HTTP POST

• XQuery Context is the Collection or Document indicated by the URI

• HTTP GET http://localhost:8080/exist/rest/db/?_query=/apple

  • Useful for simple/short queries

  • Query and Parameters are sent as URL Query String Parameters

  • Parameters must be URL Encoded

  • Variety of parameters available, e.g. _start, _howmany
    See eXist-db docs - Get Requests

• HTTP POST http://localhost:8080/exist/rest/db/

  • Useful for complex/long queries

  • Query and Parameters are sent within HTTP Request Body

  • Variety of parameters available, See eXist-db docs - Post Requests

• cURL Example

• TIP: specify _wrap=no if you want the raw results

• cURL Example

• TIP: specify wrap="no" if you want the raw results

• You can store an XQuery into the database like any other document!

• Use the Content-Type: application/xquery

• HTTP PUT http://localhost:8080/exist/rest/db/time.xq

  • cURL Example:

• Executable via HTTP GET or POST

• HTTP GET http://localhost:8080/exist/rest/db/time.xq

  • cURL Example:

• Stored Query can read HTTP Request
  e.g. request:get-parameter#2

• Consider Resources first (the things you talk about)

  • Design URI that point to Resources and Collections of Resources

• Consider Representations (of Resources) second

  • One or more representations, e.g. HTML, XML, JSON, etc.

  • Use HTTP Content-Type and Accept Headers to negotiate

• Consider actions upon those resources

  • Apply HTTP Methods as Verbs to achieve the actions

• Don't forget Hypermedia (i.e. navigability via Hyperlinks)

• There are Models for evaluating RESTful'ness

  • e.g. Richardson Maturity Model

• Consider using RESTXQ first

  • Guides you to create a "RESTful" approach

  • Enforces Statelessness

  • RESTXQ Documentation and Video: http://www.adamretter.org.uk/presentations.xml#xmlprague12

• REST Server with Stored Queries

  • Is just enough to create a REST API of your own

  • Not as pure as RESTXQ

  • REST Server Documentation: http://www.exist-db.org/exist/apps/doc/devguide_rest

• controller.xq

  • Just adds URI Routing/Rewriting to REST Server

  • URL Rewrite Documentation: http://www.exist-db.org/exist/apps/doc/urlrewrite