![](https://s3.amazonaws.com/media-p.slid.es/uploads/119594/images/1461650/Screen_Shot_2015-06-05_at_13.12.39.png)
Adam Retter
adam@evolvedbinary.com
XML Prague
@ University of Economics, Prague
2024-06-08
![](https://s3.amazonaws.com/media-p.slid.es/uploads/119594/images/4584338/gmail-logo.png)
@adamretter
Towards
RESTful XQuery
2.0
![](https://s3.amazonaws.com/media-p.slid.es/uploads/119594/images/10883746/logo-black.png)
About me
-
Director of Evolved Binary
-
UK - Software, Consultancy, Training, and R&D
-
-
Co-founder and Co-owner of eXist Solutions
-
Germany - TEI Software
-
-
Software Engineer / Prolific Open Source contributor
-
Enjoys Research and Development
-
Involved in several conference boards and peer-review panels
-
W3C XQuery Working Group - Invited expert
-
Founder of EXQuery group, and creator of RESTXQ
![](https://s3.amazonaws.com/media-p.slid.es/uploads/119594/images/6607296/eXist-book-logo-red.jpg)
What's in a title?
-
There is no XQuery version 2.0!
-
1.0, 1.1, 3.0, and 3.1
-
-
In case you were napping...
-
There was no version 1.0 of this talk that you missed!
-
It seems like only yesterday!
-
XML Prague 2012
Towards RESTful XQuery 2.0
-
Builds upon:
-
The paper: RESTful XQuery: Standardised XQuery 3.0 Annotations for REST
-
The standard: RESTXQ 1.0: RESTful Annotations for XQuery
-
-
Poses and answers the question:
Is RESTXQ 1.0 still fit for purpose today?
-
Reviews the intervening progress in:
-
XQuery standardisation
-
REST (and HTTP) Software Application Frameworks
-
Alternatives to REST
-
Feedback on RESTXQ 1.0
-
-
Proposes a set of enhancements to RESTXQ 1.0
-
RESTXQ 2.0 standard?
-
Setting The Scene
(RESTXQ 1.0)
REST - Representational State Transfer
-
Client and/or Server have some "resources"
-
e.g. documents
-
Need to exchange these (or parts thereof)
-
-
Transfer a representation of their state
-
Representation is a serialization: e.g. XML, JSON, etc...
-
Usually transferred over HTTP
-
-
No direct coupling client of client and server state
-
i.e. No Server Session object
-
-
HATEOAS
-
Simplified: Use Hypermedia (i.e. traversable links) to navigate between resources
-
-
RESTful - to adhere to REST principles!
HTTP - Hypertext Transfer Protocol
POST /catalogue/clothing/product HTTP/1.1
Accept: application/xml
Content-Type: application/xml
Host: fictional-shop.com
Connection: keep-alive
<product id="12345">
<name>A nice hat</name>
...
</product>
HTTP/1.1 201 Created
Location: http://fictional-shop.com/catalogue/clothing/product/1234
Cache-Control: no-cache
Server: Elemental/1.0.0
Date: Sat Jun 8 10:34:53 2024
Connection: Keep-Alive
Content-Type: application/xml;charset=UTF-8
Content-Length: 20
<product id="1234"/>
RESTXQ 1.0 (2012)
-
Connects XQuery and HTTP together
-
Targeted the upcoming W3C XQuery 3.0 standard (2014)
-
Targeted HTTP 1.1 (1997)
-
-
Prescribes but does not enforce RESTful principles
-
Intentionally omits any mention of a "session"
-
-
Enables building HTTP API (and Web Apps) in XQuery
RESTXQ 1.0 - Three Main Concepts:
-
A set of named XQuery Annotations
-
Added to an XQuery Function
-
Promotes it to a "Resource Function"
-
-
Two classes of annotation:
-
Resource Function Constraints
-
Resource Function Parameters
-
-
-
Additional serialization rules
-
Metadata for constructing an HTTP Response
-
-
A few XPath utility functions
RESTXQ 1.0 - Processing Model:
RESTXQ 1.0
BY EXAMPLE
(Briefly!)
RESTXQ 1.0 - Simplest Resource Function
xquery version "3.0";
module namespace my = "http://my-namespace";
declare namespace rest = "http://exquery.org/ns/restxq";
declare
%rest:path("/hello")
function my:hello() {
<hello>To all at XML Prague 2024</hello>
};
-
Path Annotation
-
(One of the Resource Function Constraint Annotations)
-
-
Matches the URI path:
/hello
-
e.g.
http://example.evolvedbinary.com/hello
-
Base URI is set by the implementing server!
-
RESTXQ 1.0 - Path Annotation
declare
%rest:path("/catalogue/{$category}/product/{$pid}")
function my:product($category as xs:string, $pid as xs:integer) {
fn:collection("/products/" || $category)/product[@id eq $pid]
};
-
Only one per Resource Function
-
Matched against the path of the URI within the HTTP Request
-
Supports simple URI Templates
-
Capture a Path Segment
-
Inject the value as a parameter to the Resource Function
-
Affords some type conversion
-
-
e.g. Matches the URI path:
/catalogue/hats/product/5645
RESTXQ 1.0 - Method Annotation
declare
%rest:GET
%rest:path("/catalogue/{$category}/product/{$pid}")
function my:product($category as xs:string, $pid as xs:integer) {
fn:collection("/products/" || $category)/product[@id eq $pid]
};
-
Zero or More per Resource Function
-
(One of the Resource Function Constraint Annotations)
-
Matched against the HTTP Method of the HTTP Request
-
Only HTTP 1.1
-
OPTIONS, HEAD, GET, POST, PUT, and DELETE
-
-
e.g. Matches GET requests for the URI path:
/catalogue/hats/product/5645
RESTXQ 1.0 - Content Negotiation 1/2
declare
%rest:PUT("{$new-product}")
%rest:path("/catalogue/{$category}/product")
%rest:consumes("application/xml", "text/xml")
function my:new($category as xs:string, $new-product as document-node(element(product))) {
xmldb:store("/products/" || $category, $new-product)
};
-
Consumes Annotation
-
(One of the Resource Function Constraint Annotations)
-
Matched against the
Content-Type
Header within the HTTP Request-
Typically used with POST or PUT Method
-
-
-
e.g. Matches:
-
PUT request
-
For the URI path:
/catalogue/hats/product
-
With the
Content-Type: application/xml
request header
-
RESTXQ 1.0 - Content Negotiation 2/2
declare
%rest:path("/catalogue/{$category}/product/{$pid}")
%rest:produces("application/xhtml+xml")
%output:method("xhtml")
function my:product($category as xs:string, $pid as xs:integer) {
fn:collection("/products/" || $category)/product[@id eq $pid]
};
-
Produces Annotation
-
(One of the Resource Function Constraint Annotations)
-
Matched against the
Accept
Header within the HTTP Request-
Often used with
%output:
serialization annotations
-
-
-
e.g. Matches GET requests for the URI path:
/catalogue/hats/product/5645
RESTXQ 1.0 - Resource Function Parameter Annotations
declare
%rest:GET
%rest:path("/catalogue/{$category}/product}")
%rest:path-param("name", "{$name-query}")
%rest:path-param("min-score", "{$min-score}", "0.25")
function my:search($category as xs:string, $name-query as xs:string*, $min-score as xs:double*) {
for $product score $score in fn:collection("/products/" || $category)/product[@id eq $pid]
[name contains text ($name-query[1] using stemming)]
where $score ge $min-score[1]
return
$product
};
-
All are optional... unlike constraints!
-
Query - HTTP URI Query string parameters
-
Form - HTML form field parameters
-
Header - HTTP Header parameters
-
Cookie - HTTP Cookie Header parameters
-
RESTXQ 1.0 - Serialization
-
Reuses XSLT and XQuery Serialization 3.0
-
Allows Serialization Parameters to be used as Function Annotations
-
e.g.
%output:method("xml")
-
-
Resource Function may return a
rest:response
element-
Must be the first item in the result sequence
-
Metadata to inform creation of the HTTP Response
-
declare namespace http = "http://expath.org/ns/http-client";
declare
%rest:path("/tea")
function my:easter-egg() {
(<rest:response>
<http:response status="418" message="Tea Time!">
<http:header name="Server" value="Tea-Bot/1.0.0"/>
<http:header name="X-With-Milk" value="Never"/>
</http:response>
</rest:response> ,
<img src="https://paradeantiques.co.uk/images/c1860-masonic-treacle-glaze-twin-spouted-teapot-01.jpg"/>)
};
Proposals for RESTXQ 2.0
(My favourites...)
Improved URI Templates in Path Annotation
-
A URI Template can now capture more than one path segment
-
Three modes of operation:
-
Simple Matching
-
Similar to RESTXQ 1.0
-
-
Basic Regular Expression Matching
-
Maps to a single Resource Function parameter -
xs:anyAtomicType
-
-
Capturing Regular Expression Matching
-
Maps to an XDM
array(xs:anyAtomicType)
type by default -
Each capturing group maps to a corresponding array entry
-
RESTXQ type conversion rules are applied to each array entry
-
Group 0, i.e. the entire matching pattern, can be optionally appended to the array
-
-
-
Optional additional type conversion information can be added
Path Annotation
-
For the path:
/product/4567/summary
-
$pid := xs:integer("4567")
-
Basic Regular Expression Matching
declare
%rest:path("/product/{$pid=[0-9]+}/summary")
function local:product-summary($pid as xs:integer)
declare
%rest:path("/animal/{$common-name}/{$sub-path=.+}")
function local:animal-root($common-name, $sub-path)
-
For the path:
/animal/cat/sleep/anywhere
-
$common-name := "cat"
$sub-path := "sleep/anywhere"
-
-
Could alternatively write:
%rest:path("/animal/{$common-name}{$sub-path=.+}")
Path Annotation
Capturing Regular Expression Matching
-
For the path:
/product/HAT9876/summary
-
$pid := ["HAT", "9876"]
-
declare
%rest:path("/product/{$pid=([A-Z]{3,})([0-9]+)}/summary")
function local:product-summary($pid)
-
For the path:
/product/COAT-123456/summary
-
$pid := ["COAT", "123456", "COAT-123456"]
-
declare
%rest:path("/product/{$pid==([A-Z]{3,})-([0-9]+)}/summary")
function local:product-summary($pid)
Path Annotation
Capturing Regular Expression Matching with Type Information
-
For the path:
/product/HAT9876/summary
-
$pid := [xs:NCName("HAT"), xs:integer("9876")]
-
declare
%rest:path("/product/{$pid(xs:NCName, xs:integer)=([A-Z]{3,})([0-9]+)}/summary")
function local:product-summary($pid)
Support for JSON
-
XQuery 3.1: An XML Query Language added support for "JSON"
-
XQuery and XPath Data Model 3.1 added Map and Array types
-
XPath and XQuery Functions and Operators 3.1 added functions for loading/parsing JSON
-
XSLT and XQuery Serialization 3.1 added JSON serialisation
-
-
RESTXQ should be updated to target version 3.1 of XQuery and associated specs.
-
JSON as the HTTP Response Body
-
Simple... Serialization 3.1 spec. has us covered!
-
-
JSON in the HTTP Request Body...
JSON in the HTTP Request Body
-
Can already be accessed from POST and PUT Method Annotations
-
RESTXQ 1.0 delivers as xs:base64Binary (or implementation defined mapping)
-
-
Add a mapping rule to all Method Annotations that take a body parameter
-
If
Content-Type: application/json
-
Parse following the rules of
fn:parse-json
-
Resource Function parameter type must be:
map(*)
orarray(*)
-
-
Controlling the parsing
-
Allow additional arguments to the body parameter name:
-
declare
%rest:PUT("{$new-product(liberal=true,duplicates=reject,escape=true)}")
%rest:path("/catalogue/{$category}/product")
%rest:consumes("application/json")
function my:hello($category as xs:string, $new-product as map(*)) {
xmldb:store("/products/" || $category, $new-product)
};
Support for Multipart
-
Before Maps and Arrays it was unclear how to efficiently support this
-
An HTTP Request or Response body can be split into distinct multiple parts
-
Each has its own set of headers and body
-
Each part may itself be a multipart...
-
-
Parse HTTP Requests with a
multipart/form-data
body -
Make accessible via Form Parameter Annotation (
%rest:form-param
)-
RESTXQ 1.0 only supported
application/x-www-form-urlencoded
:xs:anyAtomicValue*
-
RESTXQ 2.0: When parameter type is
xs:anyAtomicType
:-
Inject and convert the body of the part; file data is
xs:base64Binary
-
-
RESTXQ 2.0: When parameter type is
map(xs:string, item())
:-
Extract from the part and convert into a map containing: the content disposition parameters, headers, and body
-
If the part is itself multipart, then recurse into its body...
-
-
multipart/form-data
Multipart in the HTTP Request Body
-
Parse HTTP Requests with a
multipart/*
body-
Except
multipart/form-data
, use%rest:form-param
instead! -
Minimal compliance:
-
multipart/mixed
-
multipart/related
-
multipart/alternative
-
-
Probably only needed for HTTP POST?
-
-
Make accessible via Method Parameter Annotation (
%rest:POST
)-
The resource function parameter must accept a
map(xs:string, item())+
-
Each part is extracted and convert into a map containing: the content disposition parameters, headers, and body
-
If the part is itself multipart, then recurse into its body...
-
-
multipart/*
Multipart in the HTTP Request Body
Multipart Map Format
{
parameters: [
{
name: "my-content-disposition-parameter-name",
value: "my-content-disposition-parameter-name"
},
// For example:
{
"name": "filename",
"value": "filename if multipart/form file upload"
}
],
headers: [
{
name: "my-header-name",
value: "my-header-value"
}
],
"body": xs:string | xs:base64Binary | map(*)+
}
Multipart in the HTTP Response Body
-
Define a new Serialization Method:
rest:multipart
-
e.g.
output:method("rest:multipart)"
-
Resource Function result is a sequence of maps (optional <rest:response> first)
-
Each map is a part (of the multipart response)
-
Each map may contain additional serialization parameters
-
-
{
"rest:response" {
"output:serialization-parameters": {
"output:method": "xml"
}
},
parameters: [ ... ],
headers: [ ... ],
"body": xs:string | xs:base64Binary | map(*)+
}
Further Proposals in the Paper
-
Options for Backwards Compatibility
-
Support for HTTP Patch
-
Support for any HTTP Method
-
Improved HTTP Header Parsing
-
Support for Server Side Quality Factors in Content Negotiation
-
Inclusion of Quality Factors in Matching Resource Functions
-
Serialization Parameters within the Response
-
Support for Handling XQuery Errors
Still to think about...
-
A possible Map representation of
rest:response
-
Support CSV in HTTP Request Body
-
Server Sent Events
-
Web Sockets
-
Code Generation of RESTXQ XQuery Module stub from OpenAPI
-
Development and publication of a clear and complete standard for RESTXQ 2.0
Questions?
-
Accompanying Paper (2024):
Towards RESTful XQuery 2.0 -
RESTXQ 2.0 specification source code (NOT STARTED):
RESTXQ 2.0: RESTful XQuery -
RESTful XQuery Paper (2012):
RESTful XQuery: Standardised XQuery 3.0 Annotations for REST
(YouTube Video) -
RESTXQ 1.0 specification:
RESTXQ 1.0: RESTful Annotations for XQuery
Thank You
Towards RESTful XQuery 2.0
By Adam Retter
Towards RESTful XQuery 2.0
XML Prague @ University of Economics, Prague 2024-06-08
- 107