A Short Tour

This tour takes just a few minutes to cover the full cycle of application persistence - from a runtime data value to file storage and back again. One of the more difficult examples of data to deal with - for any persistence solution - are UDTs, or user-defined types. This section takes the opportunity to demonstrate the declaration of a UDT and any interesting details relating to its persistence.

Registering Application Types

The first step is to register an application type. A simple example appears below:

import uuid
import ansar.encode as ar

class Job(object):
    def __init__(self, unique_id=None, title='watchdog', priority=10, service='noop', body=b''):
        self.unique_id = unique_id or uuid.uuid4()
        self.title = title
        self.priority = priority
        self.service = service
        self.body = body

ar.bind(Job)

All members of the Job class are given default values and the class is registered using the bind() function. After default creation, i.e. j = Job(), all members of j contain valid data. The bind() function relies on this property to determine the types of the individual members.

Note

The bind variable exists as a shortcut to bind_message(). The reasons for this arrangement are outside the scope of this document. Suffice to say that there are further related libraries which extend the capabilities of bind().

Classes can include much more than a few int and str members. An example of what can be achieved with this style of declaration can be found here and refer to Basic Python Types for a summary table. To discover how sophisticated the type system actually is, go to More About Types and then Really Complicated Documents.

Write An Object To A File

Writing an object into file storage is most conveniently carried out using the File class:

f = ar.File('job', Job)
j = Job()

f.store(j)

The call to store() creates or overwrites the job.json file in the current folder. The contents of the file look like this:

{
    "value": {
        "body": "",
        "priority": 10,
        "service": "noop",
        "title": "watchdog",
        "unique_id": "8b2eb3c3-1057-4f96-a6cf-08eb3fae1fda"
    }
}

The file contains an instance of a JSON object and the Job object appears as the value member within that object. Other members may appear alongside the value member as the situation demands.

Note

The Job class being passed to the File object is one example of a type expression. Refer to Type Expressions for the full scope of what that parameter can be.

Support Of Other Encodings Like XML

XML is supported as an alternative encoding. Adopting XML for storing and recovering of jobs requires a single additional parameter:

f = ar.File('job', Job, encoding=ar.CodecXml)
j = Job()

f.store(j)

The contents of the new file look like:

<?xml version="1.0" ?>
<message>
  <message name="value">
    <string name="unique_id">8b2eb3c3-1057-4f96-a6cf-08eb3fae1fda</string>
    <string name="title">watchdog</string>
    <integer name="priority" value="10"/>
    <string name="service">noop</string>
    <string name="body"></string>
  </message>
</message>

Use of the XML encoding comes at the cost of increased consumption of resources. XML can consume 2-4 times as much file space as JSON and the increase in consumption of CPU cycles is typically worse. For these reasons the JSON encoding is defined as the default.

Warning

The XML encoding does not deliver the same capabilities as the JSON encoding, specifically in representation of non-printing values within byte values. Look here for details. This is a limitation of XML rather than the library.

Note

By default the store() method - and its sibling recover() - auto-append an extension to the supplied file name. This behaviour is consistent throughout the library and especially significant when dealing with collections of files inside folders. The behaviour can be disabled using decorate_names=False which returns complete responsibility for file names back to the caller.

Reading An Object From A File

Reading an object from file storage is also carried out using the File class. In fact, we can re-use the same instance from the previous sample:

j, _ = f.recover()

This results in assignment of a fully formed instance of the Job class, to the j variable. Details like the filename and expected object type were retained in the f variable and re-applied here.

The recover() method specifically returns a 2-tuple. The first element is the recovered instance and the second element (i.e. the underscore) is a version tag. Use of the underscore above obviously discards whatever that version information might have been.

As far as version management goes, this behaviour is fine in the earlier stages of software development and can also be adopted as a long term strategy with respect to versioning. If there is any chance that an application may eventually execute within an environment that does include applications with version management in use, a small check should be added to the reading code:

j, v = f.recover()
if v is None:
   normal_processing()
else:
   no_version_support()

The lack of any information in v (i.e. a None value) means “nothing of note happening here”. If this code ever encounters a file written by an application with version management enabled, the v variable will contain a version tag such as "1.7"; simpler applications with no intention of implementing version support are best advised to abandon the processing of the versioned file. The most common scenario is that the file has been created by a future version of the self-same application. In that future, version management is in use.

Note

Version detection is hard-wired into the library. An entire section is devoted to the proper use of the version information returned by the recover() method. For the moment it is enough to know that operation without actual version support is okay and how to back out gracefully when versioned materials are detected.

A Few Details

The operational behaviour of the File class can be influenced by passing additional named parameters. The full set of parameters are:

  • name

  • te

  • encoding

  • create_default

  • pretty_format

  • decorate_names

The first two are required and often all that is needed - the remainder have default values satisfying most expectations. The create_default parameter affects the behaviour of the recover() method, where a named file does not exist. If set to True the method will return a default instance of the expected type, rather than raising an exception. By default, file contents are “pretty-printed” for readability and to assist direct editing. Efficiency can be improved by setting this parameter to False. Lastly, setting the decorate_names parameter to False disables the auto-append of an encoding-dependent file extension, e.g. .xml.

Note

This library does a lot of work so that the developer doesn’t need to. Just in case a developer falls into a poor pattern of use, it can be worthwhile considering this.

Summary

That is a complete introduction into how the library should be used to implement application persistence. The significant steps were:

  • registration,

  • storing,

  • and recovering.

From a developer’s point of view this pattern of typical use demonstrates a low intellectual overhead. It also quietly delivers:

  • type sophistication,

  • human-readable representation of common types such UUIDs,

  • creation of fully-formed application types,

  • and production-ready version support.