Key Dictionaries

Key Dictionaries

This page documents the KeysDictionary class in the module opscore.protocols.keys.

A keyword dictionary collects together a set of specifications of valid keywords for use in command or reply validation. Each keyword specification takes the form of a Key validator object and they are combined into a named dictionary using, for example:

KeysDictionary("tcc",(1,2),
    Key('AirTemp',
        Float(units='C',strFormat='%.1f'),
        help='Temperature of the outside air. Used for refraction correction.'
    ),
    Key('AxePos',
        Float(units='deg',strFormat='%.3f',reprFormat='%.6f',invalidValues='NaN')*3,
        help='Actual mount position of azimuth, altitude and instrument rotator.'
    ),
    ...
)

The initial two parameters of the KeysDictionary are the dictionary name (for reply keywords, this should be the actor's name) and the dictionary version in the form of a (major,minor) tuple.

Any change to the dictionary should be reflected in a new version number. An update to the major version number implies that the new dictionary is not backwards compatible [TODO: be precise on which changes break the schema].

The dictionary name must be lower case. This is more restrictive than just performing case-insensitive name matching and is required since dictionary names are mapped into the python module name space which is generally case sensitive. Attempting to create a dictionary with a name that is not lower case will raise a KeysDictionaryError.

Each key must have a case-insensitive unique name within the dictionary. By default, keys are registered using the corresponding keyword name:

Key('name',...)

In case variations of a single keyword name must be registered within the same dictionary (for example, if two commands use the same keyword name for different purposes), then they can be assigned case-insensitive unique dictionary names using:

Key('name',...,unique='cmd1.name')
Key('name',...,unique='cmd2.name')

Keys can be retrieved from a dictionary by name:

kdict = KeysDictionary(...)
if 'name' in kdict:
    key = kdict['name']

Note that the name used above does not need to in a canonical upper/lower-case form but that matches are case insensitive.

Dictionary Distribution

Each actor specifies its valid reply keywords and generates reply messages that are distributed to various listeners. In order to validate incoming replies, a listener needs a dictionary of the generating actor's reply keywords. The actor and listener share a single keyword dictionary that exists as a file in a standard location on the python module search path, namely in the actorkeys package. The contents of this file are exactly the same as would be used inside a python program that has already imported all symbols from opscore.protocols.keys and opscore.protocols.types. For example:

KeysDictionary("tcc",
    Key('AirTemp',(1,2),
        Float(units='C',strFormat='%.1f'),
        help='Temperature of the outside air. Used for refraction correction.'
    ),
    Key('AxePos',
        Float(units='deg',strFormat='%.3f',reprFormat='%.6f',invalidValues='NaN')*3,
        help='Actual mount position of azimuth, altitude and instrument rotator.'
    ),
    ...
)

The actor and listener both load this dictionary by name using:

kdict = KeysDictionary.load("tcc")

In case the filename containing the dictionary does not match the dictionary name stored within the file, a KeysDictionaryError will be raised.

Note that there is no requirement that an actor be implemented in python, as long as it provides a reply keyword dictionary in this format (which happens to be valid python). However, a listener that is not implemented in python will need to implement its own dictionary reader (which is probably more trouble than it is worth).

Dictionary Validation and Debugging

The opscore package includes two programs under bin/ that are useful for checking that you have a valid dictionary and debugging dictionary problems. Running setup opscore will add these programs to your path automatically.

The first program scans the actorcore package where all dictionary files are kept and checks each one. It also connects to the running archive server to compare the dictionary versions in your local actorcore with those being used in production (according to this snapshot which reflects the dictionaries the archiver knew about when it was last started). You normally run this program using:

dictionaryValidate.py

By default, the production actors are obtained from the archiver server at sdss-archiver.apo.nmsu.edu but you can pass a different hostname (with optional port number) on the command line, e.g.

dictionaryValidate.py localhost:8080

In case you are running offline and just want to check the syntax of your local dictionaries, use:

dictionaryValidate.py --offline

The program lists the dictionary status of all known actors which will be one of the following:

has no dictionary: the archive server does not have a dictionary registered for this actor (this is normal for the auth and keys actors)
already in use and unchanged: your local dictionary matches the production version
X.Y replaces N.M: your local dictionary is newer than the one in production and has a larger version number
is new: your local dictionary is for an actor that has no dictionary in production now (most dictionaries will have this status in the --offline mode)
has invalid version: your local dictionary version is older than the one in production (normally an svn update in actorkeys will fix this, or else you made a mistake in updating the version)
has changed and needs a version bump: your local dictionary is different from the one in production but still has the same version (you should normally increase the minor version number by one in this case)

Note that matching is determined by a file checksum in addition to the major and minor version numbers. In case an actor has a syntax error, it will be reported, e.g.

badly formatted keys dictionary in .../actorkeys/mcp-test.py:
 >> dictionary filename and name are different: mcp-test, mcp
badly formatted keys dictionary in .../actorkeys/msg.py:
 >> name 'key' is not defined

In cases where a production dictionary is valid but is not doing what you expect for certain reply messages, you can trace how the core message parsing and validation code is handling it using:

messageTrace.py
Enter a reply message to trace how it is handled by the core parsing and validation
code. Validation is based on the current dictionaries in the actorkeys package.
Use ^C to quit this program.
>

Any messages you enter at the prompt will now be processed with verbose trace output so you can see where things are going wrong. For example, here is a successfully processed message:

> .tcc 0 tcc I TCCStatus="TTT","NNN"; TCCPos=121.000,30.000,0.000; AxePos=121.000,29.910,0.000
# Parsing...
Header: HDR(,tcc,0,tcc,I)
Keywords:
   KEY(TCCStatus)=['TTT', 'NNN']
   KEY(TCCPos)=['121.000', '30.000', '0.000']
   KEY(AxePos)=['121.000', '29.910', '0.000']
# Loading dictionary for actor "tcc"...
loaded dictionary version (3, 7)
# Validating...
found valid key {KEY(TCCStatus)=[String('TTT'), String('NNN')]}
found valid key {KEY(TCCPos)=[Float(121.0 deg), Float(30.0 deg), Float(0.0 deg)]}
found valid key {KEY(AxePos)=[Float(121.0 deg), Float(29.91 deg), Float(0.0 deg)]}

In case you are cutting and pasting from a STUI log window, leave off the initial timestamp. Use control-C to quit the program.

Dictionary Documentation

A dictionary can describe the valid keywords it contains in either plain text:

print kdict.describe()

or else in HTML format:

print >> htmlfile, kdict.describeAsHTML()

Here is some sample plain text output:

Keys Dictionary for "tcc" version (1, 2)

     Keyword: AirTemp
 Description: Temperature of the outside air. Used for refraction correction.
      Values: 1

        Repeated: once
            Type: Float (float)
           Units: C

     Keyword: AxePos
 Description: Actual mount position of azimuth, altitude and instrument rotator.
      Values: 3

        Repeated: 3 times
            Type: Float (float)
           Units: deg

     ...

Refer to the description of keyword documentation for details.