Enabling DCE Application Servers for System Management

From EDM2
Jump to: navigation, search

by Rick Cohen and Umesh Khatwani

A major consideration when you design a distributed client/server application is how to let the application server be managed remotely. A system administrator at one machine should be able to remotely monitor the activity of any server application running on the network.

You can easily enable remote management of your server application by using features that are included with IBM Directory and Security Server (DSS). For more information on DSS, refer to the article Go Cross Platform With the Developer Connection on page 1 and DSS Directory Services on page 12. The DSS features to use are the:

  • DCE Serviceability (SVC) Interface
  • DCE Control Program (DCECP)
  • DCE Event Management Service (EMS)
  • DCE Simple Network Management Protocol (SNMP) Subagent

SVC and DCECP are core features of the Distributed Computing Environment (DCE), which is provided by DSS. EMS and the SNMP subagent are installation options. By using these features, and following the steps outlined in this article, you can enable your server application to be monitored by system management applications that understand the SNMP protocol.

Overview

The DCE SVC interface provides support for messages with different severity levels and the ability to route message output according to severity. Messages can be routed to several different places, including to EMS. One additional feature is that the remote SVC interface provides the means to change the routing dynamically, while a server application is running.

Once your application enables the remote SVC interface, it can use DCECP to modify the routing specifications. Each server has its own set of routings, one routing per severity. A system administrator that wants to enable a particular SVC severity on a specific server can use the DCECP log command to modify the routing for that severity to send messages to EMS.

EMS provides a collection and forwarding point that allows messages from several servers to be collected and forwarded to System Management applications that are monitoring the application servers.

The DCE SNMP subagent provides the interface from DCE into a standard system management protocol to provide easy integration of application servers with existing system management applications.

DCE-Fig-1.gif

The DCE Serviceability Interface

Using the DCE SVC interface, you can instrument your application to send out notifications when significant events occur within the application. Each notification or SVC message can be assigned an appropriate severity level to designate importance, or level of detail. The different severity levels are:

  • FATAL
  • ERROR
  • WARNING
  • NOTICE
  • NOTICE_VERBOSE

The OS/2 DCE Application Development Guide (which you can select to install when you install DCE) recommends using the following guidelines when assigning severity levels.

  • FATAL - An unrecoverable error has occurred. The server application usually exits. An example of FATAL severity is when a server cannot complete initialization because required database files are not found.
  • ERROR - An unexpected error has occurred that is not fatal, but may limit further server operation. This severity is also recommended for requests that cannot be completed. An example of ERROR severity is when a request to delete a database object fails because the object does not exist.
  • WARNING - An error has occurred that has been corrected, or an operation has a potentially dangerous side effect. This severity is also recommended as a warning that a problem has occurred that might result in an error later. A WARNING severity could be used when a queue is almost full, which might result in an error if the queue becomes full, and no more elements can be handled by the server.
  • NOTICE - A major server event has occurred. An example of NOTICE severity is to notify that a server has completed initialization and is ready to serve client requests.
  • NOTICE_VERBOSE - A server event has occurred. An example of this message is when an entry has been deleted from a database as a result of a delete request.

The SVC interface allows the destination of messages to be specified using the SVC routing mechanism. There are several methods that a server application can use to set the routing. These include setting the routing in a system-wide routing file, setting the routing using system environment variables (for example, SVC_NOTICE, SVC_FATAL, and soon), setting the routing by calling the dce_svc_routing routine, or using the DCECP control program (discussed later in this article).

The routing mechanism also allows the specification of a variety of destinations, which include sending messages to STDOUT, to STDERR, to an ASCII file, or to a binary file. An additional routing destination sends messages to the DCE Event Management Service (discussed later). Multiple routing destinations can be specified according to severity.

In order to output a message through the SVC Interface, use the routine dce_svc_printf. This routine is similar to the C run-time printf routine, and it knows how to use the routing that was set up to send the message to the appropriate destination.

SVC also defines a mechanism to set up debugging messages. These DEBUG messages can also be routed to the locations mentioned previously.

To complete the remote server management picture, SVC provides a remote interface that you can include in your server application. The remote SVC interface enables dynamic access to the SVC routing specifications for the server application. By including this interface in your server application, you can allow the DCECP to change the server's routing specifications from a remote machine, without writing any system administration code.

The DCE Control Program

Once a server has exported the remote SVC interface, you can use DCECP to manipulate the state of the server's SVC routing. Using DCECP log commands, the routing of a server can be shown and modified. A system administrator who wants to modify the routing of a specific SVC severity on a server to send messages to EMS, can use the DCECP log modify command. The following example sets the routing for messages of severity NOTICE to go to both STDOUT and to a file called /tmp/notice.log: 

dcecp> log modify /.:/hosts/shark.austin.ibm.com/app-server \
* change {NOTICE {{STDOUT -} {FILE /tmp/notice.log}}}

The following sets the routing for FATAL messages to go to STDERR and EMS: 

dcecp> log modify /.:/hosts/shark.austin.ibm.com/app-server \
* change {FATAL {{STDERR -}{EMS -}}}

Note: /.:/hosts/shark.austin.ibm.com/app-server is the CDS name where the application server exported its SVC interface.

The DCE Event Management Service

The EMS supports asynchronous event management for use by system management applications. EMS uses the concepts of event suppliers and event consumers and sets up an event channel between them to support asynchronous communication. In the context of DCE, event suppliers can be any DCE server, and event consumers can be any application with an interest in receiving asynchronous events from one or more DCE processes. The transmission of events between suppliers and consumers is uncoupled by routing events via the EMS event channel. EMS also provides a filtering mechanism to allow administrators and consumers to control which events EMS will send.

EMS provides integration for DCE clients and servers using the DCE SVC interface and the DCE Audit interface (described in the OS/2 DCE Application Development Guide). DCE applications can use the APIs offered in SVC or Audit to become event suppliers.

The EMS APIs also support developing customized consumers that can receive event data and report on server activity. See the DCE documentation for information on how to write EMS Consumers.

The DCE Simple Network Management Protocol Subgent

The DCE SNMP subagent provides the interface from DCE into a standard system management protocol to provide easy integration of application servers with existing system management applications. It provides an SNMP interface to the system management applications to monitor various DCE resources such as DCE servers (for example, CDS server), resources (for example, clearing houses) and their statistics (for example, uptime, RPC stats) and status changes.

Based on the SNMP mode of operation, the DCE SNMP subagent listens for query requests (GET, GETNEXT, and SET) coming from the management applications, queries DCE servers for data, and finally, sends replies (GETRESPONSE) back to the management applications. Apart from providing the support for servicing queries, the SNMP subagent also provides an interface to send DCE event notifications as SNMP traps to the management applications. The events that are sent as traps from the subagent include all the DCE events of severity FATAL and ERROR, and the events specified by the application developer via a mechanism furnished by the subagent. The traps that are sent from the subagent to the management applications include information such as originating host address, time the trap was originated, event severity, and the application that generated the event.

To receive events, the DCE SNMP subagent, at startup time, registers as an event consumer with the EMS, sets up event filters to receive all events of severity FATAL and ERROR, as well as the events that were specified by the application developer. Developers can specify events of interest by adding the corresponding message indices to the file /opt/dcelocal/etc/snmptrap.tbl. These events can be of any severity. This file is part of the DCE client along with the SNMP support. It includes message indices for DCE events that are of interest to a system administrator, such as start and stop notifications emitted by the standard DCE servers. After setting up filters and registering with EMS, the subagent listens for incoming event notifications. Upon receiving the events, it converts them into SNMP traps and sends the traps to the listening management applications via an SNMP agent.

By providing a mechanism to view SVC events in coalition with EMS, the DCE SNMP subagent gives the developer and the user the ability to monitor the applications. It saves application developers considerable effort and time from having to write applications to display interesting events to the user or administrator. As for the user or administrator, it provides a user-friendly and consistent interface by interfacing with the existing system management applications through the industry-standard SNMP protocol.

Steps to Enable a Server

When designing an application server, the following steps are recommended to enable the application for system management:

  • Design event points.
  • Instrument code with calls to SVC.
  • Export the remote serviceability interface.
  • Add messages to the SNMP table.
  • Route SVC messages to EMS.

Designing Event Points

When designing the event points in an application server, the OS/2 DCE Application Development Guide recommends the following categories for event points:

  • Server Startup - Servers should output messages when they are starting up and at each significant initialization step. Servers should also identify when initialization is completed and the server is ready to receive client requests. Messages should also be output by servers when they are going down and cannot receive any more client requests.
  • ServerExit - All fatal errors that cause the server to shutdown should output a message before shutdown. The SVC subsystem has a mechanism to specify that a process should exit after a message has been output. This ensures that the fatal error was logged.
  • Impaired Efficiency - If conditions are detected that may result in inefficient operation or significant performance degradation, warning messages should be output.
  • Significant "routine" activity - Any server routine activity should be reported. This could include changing server attributes by system administrators, or database add or delete operations. These types of messages should be output with severity NOTICE or NOTICE_VERBOSE, depending on the nature of the activity.

Instrumenting Code with Calls to SVC

Once the event points have been identified, then appropriate messages are created for the events and message stanzas are put in the application SAMS file. (See the DCE documentation to find out about SAMS files and how to define messages.) A call to the routine dce_svc_printf is placed in the server code to output a message when the event occurs.

Assuming that an event has been defined to output a message when an application server has completed initialization and is ready to receive requests, the following stanza would be placed in the SVC application's SAMS file: 

start

code             XXX_s_server_listening

sub-component    XXX_s_general

attributes       svc_c_sev_notice

text             "Startup is complete, Application
                  server XXX is listening"

explanation      "Server startup has completed,
                  and server is listening."

action           "None required."

end

Notice how the severity is specified in the attributes line in the message stanza. The following sample code would appear after all server initialization is completed: 

error_status_t status; 
...

dce_svc_printf(XXX_S_SERVER_LISTENING_MSG);

rpc_server_listen( 5, &status );

if (status != error_status_ok)

dce_svc_printf(XXX_S_SERVER_LISTEN_FAILED_MSG);
...

Exporting the Remote Serviceability Interface

Exporting the SVC Remote interface is described in detail in the DCE Application Development Guide. Here is a brief summary of what is required.

To provide security for the application servers so that only users with the correct authority can change a server's SVC routing information, routines must be written with the same signature as the corresponding DCE SVC routines. These routines check whether the remote caller has the correct authority, and if so, then the dce_svc routines are called to manipulate the routing. Once these routines are written, then they are exported to the DCE RPC as the remote interface.

Adding Messages to the SNMP Table

The next step is to decide which events would be important to a system management application. As described in the section about SNMP, all FATAL and ERROR events are automatically sent to the DCE SNMP subagent. If there are additional events with other severities, then those message numbers must be entered into the SNMP subagent table, which is /opt/dcelocal/etc/snmptrap.tbl.

To find out the number of a message that you would like to put into the SNMP Trap table, you can look in one of the SAMS compiler output files. The file dcexxxmsg.h (where xxx is the SVC component name of the application server) has the mapping of the message.

Using the previous example, assume that the SAMS file is called xxx.sams. First, the SAMS compiler must be run on the SAMS file: 

sams XXX.sams

Several files are created as a result of this command. the file dcexxxmsg.h will contain an entry corresponding to the stanza above that looks like this: 

* define XXX_s_server_listening 0x11af2005

The number 0x11af2005 should go in the /opt/dcelocal/etc/snmptrap.tbl file. This means that when this message is issued via dce_svc_printf, then an SNMP trap will be generated.

Routing Messages to EMS

The final step is to add EMS to the routing for your server. To get all messages reported to EMS, the routing must be set statically when the server is initialized. This can be set using the SVC routing file, or calling the dce_svc_routing routine with a server startup parameter. For example, assume the application server is called app_server, and app_server can take multiple -w options, one per SVC severity, which specifies the routing for that severity. The startup command could be: 

app_server -w NOTICE:STDOUT:-;TEXFILE:/tmp/notice.log;EMS:-;

In your application server, the code would look like this: 

error_status_t status;
...

if (strcmp(argv [i], "-w") == 0) {

/* -w <SVC routing> */

dce_svc_routing( (unsigned char *)argv [++i], &status );

if ( status != svc_s_ok ) {

/* ERROR CONDITION */ }

}

The result of this call with the string NOTICE:STDOUT:-;TEXFILE:/tmp/notice.log;EMS:-; would route all messages with severity NOTICE to the following:

  • STDOUT
  • A file called /tmp/notice.log
  • EMS

Once the server has started, then the routing for different severities can be modified using the DCECP log commands described earlier.

Summary

Remote system management is an important consideration when designing distributed applications. Using several features offered with DCE, it is easy to enable application servers to be monitored remotely. The features to use are the DCE Serviceability Interface, the DCE Control Program, the DCE Event Management Service, and the DCE SNMP Subagent. To allow your server application to be monitored remotely, use these features and follow the steps in this article when you develop your server application.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation

Retrieved from "http://www.edm2.com/index.php?title=Enabling_DCE_Application_Servers_for_System_Management&oldid=79523"