Adam Retter

adam@evolvedbinary.com
 

XML Prague
@ University of Economics, Prague
2024-06-08


@adamretter

Towards
RESTful XQuery
2.0

About me

  • Director of Evolved Binary

    • UK - Software, Consultancy, Training, and R&D

  • Co-founder and Co-owner of eXist Solutions

    • Germany - TEI Software

  • Software Engineer / Prolific Open Source contributor

  • Enjoys Research and Development

  • Involved in several conference boards and peer-review panels

  • W3C XQuery Working Group - Invited expert

  • Founder of EXQuery group, and creator of RESTXQ

  • Me: www.adamretter.org.uk

What's in a title?

  • There is no XQuery version 2.0!

    • 1.0, 1.1, 3.0, and 3.1

  • In case you were napping...

    • There was no version 1.0 of this talk that you missed!

It seems like only yesterday!

Towards RESTful XQuery 2.0

Setting The Scene

(RESTXQ 1.0)

REST - Representational State Transfer

  • Client and/or Server have some "resources"

    • e.g. documents

    • Need to exchange these (or parts thereof)

  • Transfer a representation of their state

    • Representation is a serialization: e.g. XML, JSON, etc...

    • Usually transferred over HTTP

  • No direct coupling client of client and server state

    • i.e. No Server Session object

  • HATEOAS

    • Simplified: Use Hypermedia (i.e. traversable links) to navigate between resources

  • RESTful - to adhere to REST principles!

HTTP - Hypertext Transfer Protocol

POST /catalogue/clothing/product HTTP/1.1
Accept: application/xml
Content-Type: application/xml
Host: fictional-shop.com
Connection: keep-alive

<product id="12345">
	<name>A nice hat</name>
    
    ...
    
</product>
HTTP/1.1 201 Created
Location: http://fictional-shop.com/catalogue/clothing/product/1234
Cache-Control: no-cache
Server: Elemental/1.0.0
Date: Sat Jun 8 10:34:53 2024
Connection: Keep-Alive
Content-Type: application/xml;charset=UTF-8
Content-Length: 20

<product id="1234"/>

RESTXQ 1.0 (2012)

  • Connects XQuery and HTTP together

    • Targeted the upcoming W3C XQuery 3.0 standard (2014)

    • Targeted HTTP 1.1 (1997)

  • Prescribes but does not enforce RESTful principles

    • Intentionally omits any mention of a "session"

  • Enables building HTTP API (and Web Apps) in XQuery

RESTXQ 1.0 - Three Main Concepts:

  1. A set of named XQuery Annotations

    • Added to an XQuery Function

      • Promotes it to a "Resource Function"

    • Two classes of annotation:

      1. Resource Function Constraints

      2. Resource Function Parameters

  2. Additional serialization rules

    • Metadata for constructing an HTTP Response

  3. A few XPath utility functions

RESTXQ 1.0 - Processing Model:

RESTXQ 1.0
BY EXAMPLE

(Briefly!)

RESTXQ 1.0 - Simplest Resource Function

xquery version "3.0";

module namespace my = "http://my-namespace";

declare namespace rest = "http://exquery.org/ns/restxq";

declare
	%rest:path("/hello")
function my:hello() {
  <hello>To all at XML Prague 2024</hello>
};

  • Path Annotation

    • (One of the Resource Function Constraint Annotations)

  • Matches the URI path: /hello

    • e.g. http://example.evolvedbinary.com/hello

    • Base URI is set by the implementing server!

RESTXQ 1.0 - Path Annotation

declare
	%rest:path("/catalogue/{$category}/product/{$pid}")
function my:product($category as xs:string, $pid as xs:integer) {

  fn:collection("/products/" || $category)/product[@id eq $pid]
};

  • Only one per Resource Function

  • Matched against the path of the URI within the HTTP Request

  • Supports simple URI Templates

    • Capture a Path Segment

    • Inject the value as a parameter to the Resource Function

    • Affords some type conversion

  • e.g. Matches the URI path: /catalogue/hats/product/5645

RESTXQ 1.0 - Method Annotation

declare
    %rest:GET
	%rest:path("/catalogue/{$category}/product/{$pid}")
function my:product($category as xs:string, $pid as xs:integer) {

  fn:collection("/products/" || $category)/product[@id eq $pid]
};

  • Zero or More per Resource Function

  • (One of the Resource Function Constraint Annotations)

  • Matched against the HTTP Method of the HTTP Request

    • Only HTTP 1.1

    • OPTIONS, HEAD, GET, POST, PUT, and DELETE

  • e.g. Matches GET requests for the URI path: /catalogue/hats/product/5645

RESTXQ 1.0 - Content Negotiation 1/2

declare
    %rest:PUT("{$new-product}")
	%rest:path("/catalogue/{$category}/product")
    %rest:consumes("application/xml", "text/xml")
function my:new($category as xs:string, $new-product as document-node(element(product))) {

  xmldb:store("/products/" || $category, $new-product)
};

  • Consumes Annotation

    • (One of the Resource Function Constraint Annotations)

    • Matched against the Content-Type Header within the HTTP Request

      • Typically used with POST or PUT Method

  • e.g. Matches:

    • PUT request

    • For the URI path: /catalogue/hats/product

    • With the Content-Type: application/xml request header

RESTXQ 1.0 - Content Negotiation 2/2

declare
	%rest:path("/catalogue/{$category}/product/{$pid}")
    %rest:produces("application/xhtml+xml")
    %output:method("xhtml")
function my:product($category as xs:string, $pid as xs:integer) {

  fn:collection("/products/" || $category)/product[@id eq $pid]
};

  • Produces Annotation

    • (One of the Resource Function Constraint Annotations)

    • Matched against the Accept Header within the HTTP Request

      • Often used with %output: serialization annotations

  • e.g. Matches GET requests for the URI path: /catalogue/hats/product/5645

RESTXQ 1.0 - Resource Function Parameter   Annotations

declare
    %rest:GET
	%rest:path("/catalogue/{$category}/product}")
    %rest:path-param("name", "{$name-query}")
    %rest:path-param("min-score", "{$min-score}", "0.25")
function my:search($category as xs:string, $name-query as xs:string*, $min-score as xs:double*) {

  for $product score $score in fn:collection("/products/" || $category)/product[@id eq $pid]
      [name contains text ($name-query[1] using stemming)]
  where $score ge $min-score[1]
  return
    $product
};

  • All are optional... unlike constraints!

    • Query - HTTP URI Query string parameters

    • Form - HTML form field parameters

    • Header - HTTP Header parameters

    • Cookie - HTTP Cookie Header parameters

RESTXQ 1.0 - Serialization

  • Reuses XSLT and XQuery Serialization 3.0

    • Allows Serialization Parameters to be used as Function Annotations

    • e.g. %output:method("xml")

  • Resource Function may return a rest:response element

    • Must be the first item in the result sequence

    • Metadata to inform creation of the HTTP Response

declare namespace http = "http://expath.org/ns/http-client";

declare
	%rest:path("/tea")
function my:easter-egg() {
  (<rest:response>
  	  <http:response status="418" message="Tea Time!">
        <http:header name="Server" value="Tea-Bot/1.0.0"/>
        <http:header name="X-With-Milk" value="Never"/>
      </http:response>
    </rest:response> ,
    <img src="https://paradeantiques.co.uk/images/c1860-masonic-treacle-glaze-twin-spouted-teapot-01.jpg"/>)
};

Proposals for RESTXQ 2.0

(My favourites...)

Improved URI Templates in Path Annotation

  • A URI Template can now capture more than one path segment

  • Three modes of operation:

    • Simple Matching

      • Similar to RESTXQ 1.0

    • Basic Regular Expression Matching

      • Maps to a single Resource Function parameter - xs:anyAtomicType

    • Capturing Regular Expression Matching

      • Maps to an XDM array(xs:anyAtomicType) type by default

      • Each capturing group maps to a corresponding array entry

      • RESTXQ type conversion rules are applied to each array entry

      • Group 0, i.e. the entire matching pattern, can be optionally appended to the array

  • Optional additional type conversion information can be added

Path Annotation

  • For the path: /product/4567/summary

    • $pid := xs:integer("4567")

Basic Regular Expression Matching

declare
	%rest:path("/product/{$pid=[0-9]+}/summary")
function local:product-summary($pid as xs:integer)

declare
	%rest:path("/animal/{$common-name}/{$sub-path=.+}")
function local:animal-root($common-name, $sub-path)

  • For the path: /animal/cat/sleep/anywhere

    • $common-name := "cat"
      $sub-path := "sleep/anywhere"

  • Could alternatively write:

%rest:path("/animal/{$common-name}{$sub-path=.+}")

Path Annotation

Capturing Regular Expression Matching

  • For the path: /product/HAT9876/summary

    • $pid := ["HAT", "9876"] 

declare
	%rest:path("/product/{$pid=([A-Z]{3,})([0-9]+)}/summary")
function local:product-summary($pid)

  • For the path: /product/COAT-123456/summary

    • $pid := ["COAT", "123456", "COAT-123456"] 

declare
	%rest:path("/product/{$pid==([A-Z]{3,})-([0-9]+)}/summary")
function local:product-summary($pid)

Path Annotation

Capturing Regular Expression Matching with Type Information

  • For the path: /product/HAT9876/summary

    • $pid := [xs:NCName("HAT"), xs:integer("9876")] 

declare
	%rest:path("/product/{$pid(xs:NCName, xs:integer)=([A-Z]{3,})([0-9]+)}/summary")
function local:product-summary($pid)

Support for JSON

JSON in the HTTP Request Body

  • Can already be accessed from POST and PUT Method Annotations

    • RESTXQ 1.0 delivers as xs:base64Binary (or implementation defined mapping)

  • Add a mapping rule to all Method Annotations that take a body parameter

    • If Content-Type: application/json

    • Parse following the rules of fn:parse-json

    • Resource Function parameter type must be: map(*) or array(*)

  • Controlling the parsing

    • Allow additional arguments to the body parameter name:

declare
    %rest:PUT("{$new-product(liberal=true,duplicates=reject,escape=true)}")
	%rest:path("/catalogue/{$category}/product")
    %rest:consumes("application/json")
function my:hello($category as xs:string, $new-product as map(*)) {

  xmldb:store("/products/" || $category, $new-product)
};

Support for Multipart

  • Before Maps and Arrays it was unclear how to efficiently support this

  • An HTTP Request or Response body can be split into distinct multiple parts

    • Each has its own set of headers and body

    • Each part may itself be a multipart...

  • Parse HTTP Requests with a multipart/form-data body

  • Make accessible via Form Parameter Annotation (%rest:form-param)

    • RESTXQ 1.0 only supported application/x-www-form-urlencoded: xs:anyAtomicValue*

    • RESTXQ 2.0: When parameter type is xs:anyAtomicType:

      • Inject and convert the body of the part; file data is xs:base64Binary

    • RESTXQ 2.0: When parameter type is map(xs:string, item()):

      • Extract from the part and convert into a map containing: the content disposition parameters, headers, and body

      • If the part is itself multipart, then recurse into its body...

multipart/form-data

Multipart in the HTTP Request Body

  • Parse HTTP Requests with a multipart/* body

    • Except multipart/form-data, use %rest:form-param instead!

    • Minimal compliance:

      • multipart/mixed

      • multipart/related

      • multipart/alternative

    • Probably only needed for HTTP POST?

  • Make accessible via Method Parameter Annotation (%rest:POST)

    • The resource function parameter must accept a map(xs:string, item())+

      • Each part is extracted and convert into a map containing: the content disposition parameters, headers, and body

      • If the part is itself multipart, then recurse into its body...

multipart/*

Multipart in the HTTP Request Body

Multipart Map Format

{
  parameters: [
    {
    	name:  "my-content-disposition-parameter-name",
        value: "my-content-disposition-parameter-name"
    },
    
    // For example:
    {
      "name": "filename",
      "value": "filename if multipart/form file upload"
    }
  ],
  
  headers: [
    {
    	name:  "my-header-name",
        value: "my-header-value"
    }
  ],
  
  "body":  xs:string | xs:base64Binary | map(*)+
}

Multipart in the HTTP Response Body

  • Define a new Serialization Method: rest:multipart

    • e.g. output:method("rest:multipart)"

    • Resource Function result is a sequence of maps (optional <rest:response> first)

      • Each map is a part (of the multipart response)

      • Each map may contain additional serialization parameters 

{
  "rest:response" {
    "output:serialization-parameters": {
      "output:method": "xml"
    }
  },
  
  parameters: [ ... ],
  
  headers: [ ... ],
  
  "body":  xs:string | xs:base64Binary | map(*)+
}

Further Proposals in the Paper

  • Options for Backwards Compatibility

  • Support for HTTP Patch

  • Support for any HTTP Method

  • Improved HTTP Header Parsing

  • Support for Server Side Quality Factors in Content Negotiation

  • Inclusion of Quality Factors in Matching Resource Functions

  • Serialization Parameters within the Response

  • Support for Handling XQuery Errors

Still to think about...

  • A possible Map representation of rest:response

  • Support CSV in HTTP Request Body

  • Server Sent Events

  • Web Sockets

  • Code Generation of RESTXQ XQuery Module stub from OpenAPI

  • Development and publication of a clear and complete standard for RESTXQ 2.0

Questions?

Thank You

Towards RESTful XQuery 2.0

By Adam Retter

Towards RESTful XQuery 2.0

XML Prague @ University of Economics, Prague 2024-06-08

  • 107

More from Adam Retter