Accessing the Console/Config tree

Detailed Description

The Console/Config library provides several ways to use the node tree to configure and control an application.

1. Configuration support

The configuration support of the Console/Config library revolves around the ConfigSource concept. Each ConfigSource will somehow provide commands which will then be executed against the node tree.

To simplify the usage, there will always be three interfaces to a specific config source:

  • A constructor to build a bare config source which is then added to a senf::console::ConfigBundle (see Multiple sources)
  • A class parsing and executing a single config source. The visible interface of this class is a combination of the constructor and the senf::console::ConfigBundle interfaces.
  • A helper function which will do the complete parsing of a single source with default parameters.
When parsing these configuration sources, it is always possible to optionally change the root node used during parsing and it is also possible to restrict parsing to a command subset. See Partial / incremental configuration.

1.1. Configuration files

Constructor senf::console::FileConfig()
Class senf::console::ConfigFile
Helper senf::console::parseFile()

In it's simplest form, parsing a configuration file consists of calling senf::console::parseFile() with the name of the respective config file as argument.

senf::console::parseFile("some.conf");

To get more flexible, instantiate a senf::console::ConfigFile instance at use that to parse the file

senf::console::ConfigFile cf ("some.conf");
// The following line is optional: Call to ignore mussing files
cf.ignoreMissing();
cf.parse();

If the application supports other configuration sources besides a single configuration file (like command line options) or if it supports multiple configuration files (e.g. a system-wide and a user specific configuration file) see Multiple sources and add one (or more) senf::console::FileConfig() source to a senf::console::ConfigBundle.

1.1.1. Configuration file syntax

Configuration files are written in a simple configuration language. This language is almost declarative (e.g. it does not have any control-flow statements) but is processed imperatively from top to bottom. This is very simple and flexible.

Commands are referenced by their path in the node tree. To simplify working with deeply nested directory structures, the current directory may be changed persistently or temporarily for some commands.

/server/port 1234;

/logger/targets/console {
    accept senf::log::Debug IMPORTANT;
    accept server::ServerLog CRITICAL;
}
See also:
The parser

1.2. Command line options

Constructor senf::console::OptionsConfig()
Class senf::console::ProgramOptions
Helper senf::console::parseOptions()

Command line options can either be parsed by calling the senf::console::parseOptions() helper

senf::console::parseOptions(argc, argv)

or more flexibly by instantiating a senf::console::ProgramOptions class

std::vector<std::string> args;
senf::console::ProgramOptions opts (argc, argv);
opts
    .nonOptions(args)
    .alias('c', "--mycommand",true)
    .alias('C', "--mycommand=2 3");
opts.parse();

This registeres two short options and accumulates all non-option arguments in args.

If the application supports other configuration sources besides the command line options (like configuration files) see Multiple sources and add a senf::console::OptionsConfig() source to a senf::console::ConfigBundle.

See senf::console::ProgramOptions for the source specific additional parameters. These apply to senf::console::ProgramOptions and to the senf::console::OptionsConfig() source.

1.2.1. Options syntax

Command line options are primarily parsed as long-options. Long options start with '--'. Further '-' characters serve as directory separators if required (that is, they are only interpreted as directory separator is there is no entry in the current (sub-) directory matching more than a single name component). This still allows using hyphens in node names.

Options can be abbreviated at each directory boundary: A command /foo/bar/do can be called as --f-b-d as long as this name is unique.

Everything after the first '=' character is passed as arguments to the command. The exact interpretation depends on the command:

  • If the command only takes a single token as argument (e.g. a single string or numeric value), everything after the '=' sign is parsed into a single token (e.g. see rows 2 and 3 of the following table).
  • In all other cases, the string after the '=' sign is parsed into argument tokens using the config/console parser. In this case, quoted strings need to be quoted twice, once for the shell and once for the config/console parser (e.g. see rows 4 and 5 of the following table).
  • If the option has no '=' character, the list of argument tokens will be empty (e.g. see row 1 of the following table)
Without these rules, multi-word string arguments would always have to be quoted twice (for the shell and the config/console parser).
Command File syntax Option syntax
void doo() /path/to/doo; --path-to-doo
void doo(std::string const &) /path/to/doo john.doe@everywhere.org; --path-to-doo="john.doe@everywhere.org"
void doo(std::string const &) /path/to/doo "some text"; --path-to-doo="some text"
void doo(std::string const &, int) /path/to/doo take 1; --path-to-doo="take 1"
void doo(std::string const &, int) /path/to/doo "take two" 1; --path-to-doo='"take two" 1'

Short options are registered as aliases for long options. They can be registered with or without an implied parameter and can optionally take a parameter. so after

opts
    .alias('c', "--mycommand",true)
    .alias('C', "--mycommand=2 3");

we can call

$ program -C -c "4 5"
$ program -Cc"4 5"

which is the same as

$ program --mycommand="2 3" --mycommand="4 5"

(Beware, that the second argument to alias() must not be shell quoted).

1.3. Changing the root node

When used in it's default state, parsing will always interpret all commands relative to the senf::console::root() node and will parse a file completely.

The first possibility to control this is to change the root node. This is done by

  • passing that root node to the helper class or to the parse helper as an additional argument (see the respective documentation).
  • passing it to the senf::console::ConfigBundle constructor when parsing multiple sources.
for example: 
senf::console::parseFile("/etc/myserver.conf", senf::console::root()['config']);

This functionality is even more powerful by combining it with link nodes: This allows to selectively choose commands from the node tree which are to be made accessible for configuration. See The node tree.

1.4. Partial / incremental configuration

Another feature provided by senf::console::ConfigBundle and all helper classes is partial parsing. 
// Create a console/config aware object and place it (that is it's directory node) into the node
// tree
FooObject foo;
senf::console::root().add("foo", foo.dir);

// Open configuration file
senf::console::ConfigFile cf ("/etc/myserver.conf");

// Parse only commands in the configuration file which are in the foo.dir directory
cf.parse(foo.dir);

...

// Anywhere later, parse the rest of the configuration file
cf.parse();

This feature allows to parse parts of one or more configuration sources before the console/config tree has been fully established. Partial parsing can be applied any number of times to arbitrary nodes. Any command already parsed will be skipped automatically.

When combining partial parsing with chroot() and link's, it is important to realize, that partial parsing always applies to the real target and ignores links. This is very important: It allows a subsystem to parse it's configuration parameters irrespective of any links pointing to nodes of that subsystem.

1.5. Multiple sources

Most of the time, an application will utilize multiple configuration sources: A global configuration file, maybe a user specific local configuration file, command line options ...

When parsing configuration commands, especially using partial / incremental parsing, all parse commands should be applied to each configuration source in turn. This is the responsibility of senf::console::ConfigBundle.

senf::console::ScopedDirectory<> config;
senf::console::root().add("config", config);

// Let's enable all logger commands for configuration
config.link("logger", senf::console::root()["logger"]);

// Create bundle and add sources
std::vector<std::string> args;
senf::console::ConfigBundle conf (senf::console::root()["config"]);
conf.add( senf::console::FileConfig("/etc/myserver.conf") );
conf.add( senf::console::FileConfig(".myserver.conf")->ignoreMissing() );
conf.add( senf::console::OptionsConfig(senf::Daemon::instance().argc(),
                                       senf::Daemon::instance().argv()) )
    .nonOptions(args)
    .alias('c', "--mycommand",true)
    .alias('C', "--mycommand=2 3");

// Parse the logger subsystem commands in '/logger'
conf.parse(senf::console::root()['logger']);

...

// Parse all other configuration commands. All necessary commands and links in '/config' must by
// now have been created.
conf.parse();

This example parses three configuration sources: Two configuration files and additional parameters specified on the command line. All the configuration commands are placed into the /config directory (directly or via links). The configuration sources are parsed in the order they are specified, so in this case, the command line options will override any options specified in one of the configuration files.

2. The network console

To make the network console accessible, it must be initialized when the program is started: 
#include <senf/Console.hh>

int main(int argc, char * argv [])
{
    // Configure console nodes, add commands ...

    // Start console server
    senf::console::start(senf::INet4SocketAddress(12345u))
       .name("myserver");

    // You need to enter the scheduler main-loop for the server to work
    senf::scheduler::process();

    // Alternatively enter the main-loop via the PPI
    // senf::ppi::run();
}

This will start the server on IPv4 port 12345. The servers name (as displayed in the interactive console prompt) is set to 'myserver'.

After launching the application, the server can be accessed at the given port:

bash$ telnet localhost 12345
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

myserver:/$ exit
Connection closed by foreign host.
bash$

It is possible to start multiple server consoles by calling start() multiple times with different ports/addresses. Each server can be configured separately (e.g. root node, mode ...).q

2.1. Server and Client objects

The senf::console::Server and senf::console::Client objects offer further API calls. To access the server instance you need to store away the senf::console::Server reference returned when starting the server so you can later refer to it: 
int main(int, char**)
{
    senf::console::Server & server ( senf::console::start( ... ) );

    // Do something ...

    server.stop()
}

The client instance can be accessed via the std::ostream arg of any command callback

void someCallback(std::ostream & os, ... )
{
    senf::console::Client & client (senf::console::Client::get(os));

    // Use the client's log target
    client.route<senf::log::Debug, senf::Log::IMPORTANT>();
}
See also:
senf::console::Server for the Server API
senf::console::Client / List of all members for the Client API

2.2. The interactive console shell

The interactive shell implements a fully function line editor on capable terminals. This support is available when using a full featured telnet client on a fully supported terminal (like vt100 or xterm).

The shell supports auto-cd and auto-completion: If you enter the name of a directory at the prompt, the console will change to that directory. With auto-completion, any unique beginning of a path component will be completed automatically and transparently to the corresponding full name.

2.3. Non-interactive network console

After a new connection is established, the console server waits a short time for data to arrive. Only if nothing happens in the first 500ms, an interactive session is initialized.

By sending data immediately after opening the connection, the console is switched into non-interactive mode. In this mode, no prompt is displayed. In this mode, commands are not terminated automatically by end-of-line (CR). This allows, to easily cat an arbitrary configuration file into the network console using netcat:

$ nc -q1 localhost 23232 < some.conf

The argument -q1 makes netcat close the sending end of the connection on EOF and wait up to 1 second for the console to terminate. Even better, use netcat6, which has full TCP half-close support.

$ echo "ls" | nc6 --half-close localhost 23232 2>/dev/null
console/
server/
test/
$

Commands are executed as soon as the terminating character (';', '{' or '}') is received or when the sending end of the connection is closed.

3. Non-interactive UDP console

The UDP console allows to script the console tree via UDP packets. Every UDP packet must be a complete command (or sequence of commands). The combined reply of all these commands will be returned in a single UDP packet. This reply can be disabled or directed to a different address.

To start a UDP server, just create an instance of the senf::console::UDPServer class

senf::console::UDPServer server (senf::INet4SocketAddress("127.0.0.1:23232"));
(Remember to enter the scheduler main-loop for processing)

Commands may then be sent to this UDP console e.g. using netcat

$ echo "cd sys; ls" | nc -uq0 localhost 23232 2>/dev/null
See also:
senf::console::UDPServer

4. Classes

class   senf::console::ConfigBundle
  Combine multiple configuration sources. More...
class   senf::console::ConfigFile
  Console node tree based config file parser. More...
class   senf::console::ProgramOptions
  Console node tree based command line option parser. More...
class   senf::console::Server
  Interactive console server. More...
class   senf::console::Client
  Server client instance. More...
class   senf::console::UDPServer
  UDP Console server. More...