NETPP Documentation

Version:
0.30
Author:
Martin Strubel
Date:
01/2011

Introduction

The Device Control library (DCLIB) is a library of tools and C source code designed to remote control properties of devices and quickly prototype access to registers on an embedded device. The NETPP enhancement of this library is a network variant of the DCLIB where most of the intelligence is outsourced to the device controller, therefore avoiding raw register protocols between the embedded devices.

A NETPP capable device in a network can communicate its properties to a client in a flexible way. A client can talk to devices of completely different architecture, it is up to the client software to check the device's capabilities according to whether a property exists, or according to a user defined device type field.

To keep the overhead at a minimum for embedded targets, properties are defined as a static array of structures. Those structures are not spelled out manually in source code, but generated from an XML description. The XML description supports several data types (INT, BOOL, BUFFER, etc.) as well as different access methods (global variables, handler functions, etc.)

In short, a few more properties of the netpp library:

Note that netpp does not cover 16 bit TOKEN size anymore. Since most compilers support 32 bit values ('long'), this is not a big issue. Optionally, a non-network variant can be built using specific binary protocols.

Client and Server programs

To outline the functionality of NETPP: A typical application will consist of a server running on the device to be controlled, and a frontend on the client PC to talk to it. This is in general referred to as Master (PC, User) and Slave (responding Device). A Slave will, under normal circumstances, never initiate a connection.

To summarize the terminology:

NETPP provides two rudimentary frontends:

  1. A example test program, allowing to browse the property hierarchy, getting and setting of properties. See master/ folder.
  2. A python module, allowing scripted remote control and easy system testing. See python/ folder.

The example program's source code is documented under the Examples section (see master::c)

API Details

The API interface uses the standard C convention (CDECL) interface. NETPP is portable among different operating systems such as Microsoft Windows, Linux, QNX, and other Unix versions.

All Master functionality is hinted with the prefix 'dc' in the listed API functions. The Slave functionality is not completely documented, please see Implementing a NETPP server backend on Slave and Implementing a new NETPP driver / server for hints and examples.

Property concept

The architecture also allows to integrate user defined children types to a data type via structs that allow signalling events to a specific GUI element when a Property was changed, etc.

To keep the API and the library code as compact and efficient as possible, the Property access happens via tokens. The token is obtained by dcProperty_ParseName() via a given name, which is defined in the name space of a certain parent node. The top root node of a Property hierarchy is always the device node which is obtained by dcDevice_GetRoot().

Value passing is done via a runtime data type information capable structure 'DCValue'. This structure contains a data union field and an associated data type. The value is retrieved from the DCValue struct according to the type field. The Property getter and setter functions of the library perform a type check and return an error (DCERR_PROPERTY_TYPE_MATCH) in case of a mismatch.

Strings and buffer structures are mostly owned by the caller and are not allocated statically by NETPP, unless explicitely stated. When obtaining a buffer, the caller always must preinitialize the DCValue with a pointer to the reserved buffer and the length value to prevent buffer overflows. This is done as follows:

unsigned char buffer[32];
DCValue v; v.value.p = buffer; v.len = sizeof(buffer); v.type = DC_BUFFER

Then, a dcDevice_GetProperty() or dcDevice_SetProperty() can take place. For more details, see example master::c.

When a string or buffer is allocated static (for example, a port descriptor or other local property structures), v.value.p does not need to be preinitialized. It points to the static buffer location of the requested property after a successful call. So, it is in general not a good idea to assign v.value.p to a malloc() return value directly, as it may be modified within a GetProperty() call.

If the data type is not known a priori, v.type can be set to DC_UNDEFINED. The handler then fills in the data type.

If the buffer size can be dynamic on the device side, it may be desired to query the buffer size beforehand. This is done via 

DCValue v;
retcode = dcProperty_GetProto(device, T_property, &v);

Note: This does not work with buffer properties that are implemented via a handler (non static buffers). This is a current limitation of the library. In case of a handled (dynamic) buffer, v.len contains 0 for a buffer or 1 for a string.

Common function parameters

Generally, the first parameter of a function whose name starts with 'dcDevice_' is the current device device handle. The struct members of the device object must NEVER be accessed directly, to preserve future binary compatibility.

If not documented otherwise, the function's return value is

See also:
Error Handling

Functions which set a value may return DCWARN_PROPERTY_MODIFIED, meaning that the given parameter was not valid or not according to certain constraints. All 'Event' children of this property should be queried via dcProperty_Select(), because they may have changed. In this case, the user should also read back the value that was passed to dcDevice_SetProperty() which was possibly modified 'in place'.

Error codes are split in library specific and device specific error codes. Typically, an application controlling a backend includes "devlib_error.h" (containing the generic library error codes) and "DEVICE/device_error.h" (device and backend specific error codes). This file is normally found in the device specific directory netpp/devices/DEVICE (where DEVICE is to be replaced by the device implementation name). It is recommended, to set the include paths as follows: 

INCLUDES = -I$(NETPP)/include -I$(NETPP)/devices

The function dcGetErrorString() returns the library specific (device independent) error codes. Normally, an application first checks for device errors and then calls this function to get a verbose error message. To create a user defined error handler, you may want to derive an automatic generation from $NETPP/xml/errorhandler.xsl.

Property Implementation

Property definition

The property definition happens in XML according to a schema. A property always has a type specification and an access description. To map a device side variable, flag, or buffer into a property, there are several approaches.

For the above, please refer to the XML implementation in devices/example/example.xml

To write the XML file, it is recommended to use an XML editor like XMLMind Xml Editor, see http://www.xmlmind.com. The provided schema files and style sheets are especially customized for this editor. See an example XML file below.

xxe.png

XMLMind XML Editor example

The style sheet is customized for the XMLMind XML Editor (XXE) such that the item hierarchy is visible in coloured blocks. The underlying Schema files guarantee that the syntax is always correct and that the hierarchy is built according to the schema rules.

Property types

As of now, the following property types exist:

Special properties

Properties can be part of a container such as a Struct or Array node. In the XML description, they display like a property, but contain other properties. A Struct node is a very simple container, it is just a local name space for properties. For example, you would access a property inside a struct via the name "VideoStream.Enable". The members of a struct may be restricted by the schema file to a certain number. This limitation can be changed by the user.

Array properties can either contain:

Currently, only 1-dimensional arrays are supported for register addressing mode. In some implementations, two dimensional arrays may be supported via handlers.

The size of an array can always be queried via the special child 'Size', e.g. "Coordinate.Size". The second children of an array node is always the contained object. Its name does not matter, since it is addressed by the index automatically.

Structures in Arrays

Assume, you have a register table with values [X, Y, W, H, Mode] somewhere in your chip address space. You'd want to save these values in a non-volatile space on the chip for fast context switching. So you would have a copy of this table somewhere else in memory. This lists in short, how you wrap this with a struct/array/property hierarchy:

  1. Create an array node and a struct node as child. Inside this struct, you define the member properties according to the register table.
  2. Add a 'array base' register to your register table. The 'size' attribute of this register defines the field size inside the array, like the address difference (not necessarily in bytes) from element[0] to element[1]. The offset address should be set to 0x0
  3. Refer to this base register from within the array node via a 'regref'.
  4. For each struct member, create a register entry using the absolute address of the struct member in the first array element.
  5. Refer to this register definition using a 'regref' node within the struct property member

An example is found in the LUTvalues array property in devices/example/example.xml

If the structured register tables are organized contiguously (respectively, using a fixed address spacing between the tables), they can be simply addressed like "Context[index].Member".

Translation to source code

The XML device description is translated to source code via the XML processor 'xsltproc', which must be installed separately from netpp. It is part of every standard Linux distribution or Cygwin distribution on Microsoft Windows.

The generation of the source code happens according to XSLT style sheets that are residing in the xml/ sub folder of the netpp distribution.

scheme.png

XML conversion scheme

A makefile will do the following, by running it from the build directory via 'make':

Optionally, the following type of files can be generated from the XML device description:

See devices/ for example devices and their makefiles.

String and buffer handling

On device side, an implementation of a buffer property basically consists of an address and length field. The 'Property Engine' is responsible of tunneling the data through a specific protocol with minimum buffer copy overhead.

When a variable specification is used, the engine knows the address immediately from the specification. The only thing the caller, i.e. remote control host needs to do, is to reserve enough memory for the string or buffer data, see Property concept. With a handler, things can get more complicated. First, the handler needs to do a check on the buffer size and return a DCERR_PROPERTY_SIZE_MATCH, if the callers buffer is too small. Or, adapt to the situation and send data in chunks. The handler must also pass on the buffer address to the engine before the transfer. When the transfer is finished, the handler is called again with a DCValue type field set to DC_COMMAND. This signals, that the buffer was transferred and an update action can take place, for example when an image is to be redrawn. See also example handler::c set_buffer().

This scheme is applying to buffer reads and writes. For a setter, the handler is notified about the transfer termination (all data arrived) Likewise, the getter is informed that all data was transferred and that the owner may release the allocated buffer space.

Variable changes, events

In many devices, settings of a property can have impact on another. For example, changing the readout width of a optical sensor may affect the timing. Or very simple, issueing a Reset command on the target requires some registers being reread by the client to synchronize with the user interface.

To address this issue, an event node has to be inserted into the property. When the handler returns DCWARN_PROPERTY_MODIFIED, the event children must be polled as well. An event node simply contains a 'id' reference to another node (which currently must be a property).

For more complex, dynamic rebuilds, you have to use your own mechanisms on a higher level within netpp.

Event properties are static and have an 'EVT_' prefix in their property name.

Hubs and Port access

A Hub represents a specific interface respectively transport layer, for example USB or TCP. It is for example possible, to control the same device Slave via different interfaces. Hubs are instanced as dynamic property nodes, similar to the static property lists, so they can have children. A Port again is always a children of a Hub and represents an attached device that can be opened.

Normally, Hubs are only relevant to the master, however, a Slave uses Hubs internally for each interface it listens to. Ports do not have a meaning on the Slave side, so this paragraph applies to Master actions only.

To browse this hierarchy, a special device handle DC_PORT_ROOT exists. Actions on this pseudo device, such as dcProperty_Select() operate on the local port tree structure. See also example master::c for details. To select the first available hub, you have to call dcProperty_Select() as follows: 

TOKEN parent = TOKEN_INVALID;
error = dcProperty_Select(DC_PORT_ROOT, parent, parent, &child);

Whenever this happens, the ports are initialized and probed. Thus, an update of a device list procedure automatically rescans the available hubs for newly attached devices that way.

Implementing a NETPP server backend on Slave

For implementing a NETPP server, the following steps need to be taken:

  1. Create a device file in XML
  2. Write property handlers if needed (see Creating handlers for properties below)
  3. Implement device backend stubs/handlers
  4. Register handlers/properties in your server code

For step 1, there are a few different approaches. One scenario is, that you have a set of existing global variables that you want to export to NETPP. Another approach is, to start the property design first and then implement the variables and handlers where necessary.

As as starting point, you may want to look at the server example (see module NETPP server functionality)

Exporting Variables to NETPP

Assume, you want to export a variable x to NETPP, standing for an image width, for example. If the variable is not yet global, it is recommended to create a context structure that is globally instanced once and make this variable a member of this structure.

When wrapping this variable with a property, you will have to create a new property with a child called 'variable'. In this variable field, you enter the name of the globally accessible variable, in this example: 

g_imgbuf.x

See devices/example/example.xml

In the 'header' node of the XML file, you must enter the name of the header file include statement where the global struct is defined.

A few comments about data types: There are only a few valid data types that can be wrapped. These are:

If you want to wrap an integer type different from the above, you have to either change the variable to int or use the handler methods described below to access the variable.

Creating handlers for properties

When you implement properties that must cause an action when they are accessed, you would typically use a handler. This is a piece of C code with a uniform Getter/Setter API. When you have created a XML file with a few handlers, run the following command in the devices/<yourdevice> directory to create a handler skeleton template:

 make handler_skeleton.c

Typically, this file is manually copied to handler.c and the handlers are implemented.

When dealing with buffers, make sure you have read Section String and buffer handling about buffer update/release functionality.

VERY IMPORTANT: Make sure that your program does not spend too much time inside the handler. For example, if the handler waits for a buffer for a longer time than the protocol timeout, the protocol breaks and resets to idle state. So make sure, that your handler processes fast enough. If it does not, you must split up the request for a buffer and the actual transfer into separate property commands, or use a specific error code saying that the buffer is not yet ready. For example in case of a very slow camera, you would use a Trigger property to start the process, poll the Trigger property again to check whether the command has completed and then finally read the buffer.

Special properties

Derivation of devices from base class

When using a lot of similar devices, it makes no sense to specify properties over and over again. This makes a class of devices hard to handle, when a basic feature is to be changed.

Therefore, NETPP is able to derive from a base class. This is simply done by adding another "device" specification to the XML file and setting the 'baseclass' attribute to the base class device's id field.

See devices/example folder on how this is done.

On target / backend side, you will need to implement a function called local_getroot() that returns the root node of the selected device. Normally, this returns TOKEN 0 when only one device is implemented. It is up to this function to determine what device description instance is to be used from the definition. This allows a certain device class to determine its properties at runtime. The user thus has the freedom to implement a device control in the following ways:

Overriding of specifications

Variables that are both specified in the base device and derived device's name space, are overridden by the definition in the derived class. So, in the above example, property 'Test2' is defined in both classes as INT but with different static value. When accessing 'Test2' by name, you will always see the value from the device "DerivedDevice". However, when looking at the property list, the property 'Test2' will appear twice. So it is possible to still access the old property by its TOKEN, however, this needs to be determined by walking through the hierarchy with dcProperty_Select().

Implementing a new NETPP driver / server

For developing a new driver, check existing sources, such as unixdevice.c Basically, you will have to implement a new struct protocol_methods for the new interface. Also, you will have to define a server init function which initializes a Hub structure. Since the driver model tries to be as symmetrical as possible, there are not separate functions for the master and the slave, unless Master and slave are using a completely different operating system. However, it may help to keep the differences between master and slave into account:

Slave side

To develop a new NETPP server for a device, you might want to start from existing code such as in devices/example.

In the main loop of your server (or your own main loop) you will have to repeatedly call the slave_handler() to listen to your interface.

Master side

See protocol_methods which methods are relevant for the master only. In ports_init() you will have to initialize a Hub for the new driver and append it to the global PortBay using portbay_append(). See ports.c for example. Note that ports_init() is called internally, the front end must not call it.

NETPP hacking and extending

Developers who wish to modify, improve or enhance the netpp code should keep the following rules in mind:

If you think there is a reason for changing files in a "no go" area, please consult the netpp author(s). Note that changes in $NETPP/src or $NETPP/common must be published according to the netpp source license.

A few notes about the coding style:

Generated on Thu Feb 24 13:50:16 2011 for Device Control library by  doxygen 1.6.1