Using properties from within XQuery modules

Today I ran into an interesting issue. Sedna XMLDB does not support declaring external variables. But sometimes we need to have access to properties, even from a stored xquery module. We could hardcode values but some values are environment specific so no luck there.

So I was scratching my hair today and decided to get lucky on the Sedna mailinglist. Charles Foster was kind enough to share some ideas and one idea was storing the properties in the XMLDB itself.

So I quickly hacked together some prototype properties xquery support library which offers similar functionality to the Cocoon-Spring-Configurator. We can specify generic properties on the highest level and also specify environment specific properties. The nice thing is we can actually always pas the environment (prod / test/ dev) and if it can't find the property it will default to searching for a generic property by that name.

<?xml version="1.0" encoding="UTF-8"?>
<properties id="test-suite">
    <!-- default properties -->
    <property id="string1">this is text</property>
    <property id="boolean_false">false</property>
    <property id="boolean_true">true</property>
    <property id="int">5</property>
    <property id="double">3.56</property>
    <property id="decimal">6.23</property>
    <property id="float">002002.270</property>
    <property id="time">12:20:46.275+01:00</property>
    <property id="dateTime">2002-12-07T12:20:46.275+01:00</property>
    <property id="date">2002-12-07</property>
    <property id="duration">P30Y243D</property>
    <property id="anyURI">http://www.google.com</property>
    <environment id="prod">
        <property id="base_uri">http://nww.prod.spider.nxp.com</property>
        <property id="port">8513</property>
    </environment>
</properties>

Properties XQuery library:

module namespace properties = "http://www.nxp.com/properties";

declare function properties:getPropertyFiles() as element(properties)* {
    collection("properties")/properties
};

declare function properties:getPropertyFile($id as xs:string) as element(properties)? {
    properties:getPropertyFiles()[@id=$id]
};

(:  properties:getProperty("test-suite.string1") :)
declare function properties:getProperty($expr as xs:string) {
   properties:getProperty($expr, ())
};

(: properties:getProperty("test-suite.base_uri", "test") :)
declare function properties:getProperty($expr as xs:string, $env as xs:string?) {
    let $tokens := tokenize($expr, "\.")
    let $fileId := $tokens[1]
    let $propertyId := $tokens[2]
    let $property := if (exists($env) and exists(properties:getPropertyFile($fileId)/environment[@id=$env]/property[@id=$propertyId]))
       then properties:getPropertyFile($fileId)/environment[@id=$env]/property[@id=$propertyId]
       else properties:getPropertyFile($fileId)/property[@id=$propertyId]
    return  if (exists($property)) then data($property)
            else fn:error(fn:QName('http://www.nxp.com/error', 'properties:doesNotExist'), concat('Property ', $expr, ' does not exist'))
};

import module namespace properties = "http://www.nxp.com/properties";

<test-suite>
  <test>test-suite.base_uri      = {properties:getProperty("test-suite.base_uri", "prod")}</test>
  <test>test-suite.port          = {properties:getProperty("test-suite.port", "prod")}</test>  
  <test>test-suite.string1       = {properties:getProperty("test-suite.string1", "prod")}</test>
  <test>test-suite.boolean_false = {properties:getProperty("test-suite.boolean_false")}</test>
  <test>test-suite.boolean_true  = {properties:getProperty("test-suite.boolean_true")}</test>
  <test>test-suite.int           = {properties:getProperty("test-suite.int")}</test>
  <test>test-suite.double        = {properties:getProperty("test-suite.double")}</test>
  <test>test-suite.decimal       = {properties:getProperty("test-suite.decimal")}</test>
  <test>test-suite.float         = {properties:getProperty("test-suite.float")}</test> 
  <test>test-suite.time          = {properties:getProperty("test-suite.time")}</test>
  <test>test-suite.date          = {properties:getProperty("test-suite.date")}</test> 
  <test>test-suite.dateTime      = {properties:getProperty("test-suite.dateTime")}</test>
  <test>minutes from test-suite.dateTime = {minutes-from-dateTime(properties:getProperty("test-suite.dateTime"))}</test>
  <test>test-suite.anyURI        = {properties:getProperty("test-suite.anyURI")}</test>
  <test>test-suite.duration      = {properties:getProperty("test-suite.duration")}</test>
  <test>year from test-suite.duration      = {years-from-duration(properties:getProperty("test-suite.duration"))}</test>
</test-suite>

Output from test-suite

<test-suite>
  <test>test-suite.base_uri      = http://nww.prod.spider.nxp.com</test>
  <test>test-suite.port          = 8513</test>
  <test>test-suite.string1       = this is text</test>
  <test>test-suite.boolean_false = false</test>
  <test>test-suite.boolean_true  = true</test>
  <test>test-suite.int           = 5</test>
  <test>test-suite.double        = 3.56</test>
  <test>test-suite.decimal       = 6.23</test>
  <test>test-suite.float         = 002002.270</test>
  <test>test-suite.time          = 12:20:46.275+01:00</test>
  <test>test-suite.date          = 2002-12-07</test>
  <test>test-suite.dateTime      = 2002-12-07T12:20:46.275+01:00</test>
  <test>minutes from test-suite.dateTime = 20</test>
  <test>test-suite.anyURI        = http://www.google.com</test>
  <test>test-suite.duration      = P30Y243D</test>
  <test>year from test-suite.duration      = 30</test>
</test-suite>

Now let's try and see what happens if we access a non existing property.

import module namespace properties = "http://www.nxp.com/properties";

<test-suite>
  <test>should result in exception = {properties:getProperty("test-suite.nonexisting", "prod")}</test>
</test-suite>

2012/09/14 09:40:17 database query/update failed (SEDNA Message: ERROR doesNotExist
    Property test-suite.nonexisting does not exist
)

Now we only need to make sure that the correct environment is passed. As all our environments use a different database we only need to store a specific constants library in each database and we're good to go.

module namespace constants = "http://www.nxp.com/constants";

declare variable $constants:ENVIRONMENT as xs:string := "test";

So now we can rewrite our little test-suite to use this constant

import module namespace properties = "http://www.nxp.com/properties";
import module namespace constants = "http://www.nxp.com/constants";

<test-suite>
  <test>test-suite.boolean_false = {properties:getProperty("test-suite.boolean_false", $constants:ENVIRONMENT)}</test>
</test-suite>

How to easily deal with timestamp based URLs

Let me shortly describe the problem. You have to fetch data from some website which exposes the data based upon timestamped URL's. For the below use case the report is generated weekly on monday.

An example {server}:{port}/exports/classificationreport_20120924.csv

So you can't exactly hardcode the URL as a property but will need to generate it dynamically whenever a request is made to that resource. I've already used joda-time in the past so I decided to use it again.

public String getClassificationReportURL() {
   MutableDateTime now = new MutableDateTime();
   now.setDayOfWeek(DateTimeConstants.MONDAY);
   DateTimeFormatter fmt = DateTimeFormat.forPattern("yyyyMMdd");
   return getExportsBaseURL() + "classificationreport_" + fmt.print(now) + ".csv";
}