) then all remote applications would be
affected. One solution is to have one server - with a well-known name -
whose only task is to start a dedicated engine on behalf of a client's
request. The engine creates a unique connection whose name is transferred
over the server connection back to the client. The client closes the
connection to the server and opens a connection to the engine.
Ok, so how is does the communication take place on function call level? An
observant reader notices that RmxCreateUnique() is not mentioned at all,
and that a function RmxGetUniqueName() seems to be used. The reason is
that initially there was no RmxCreateUnique(), but the same functionality
was obtained by first calling RmxGetUniqueName() and then RmxCreate().
While I was implementing the TCP/IP DLL I noticed that it is a lot simpler
to combine the two. I just hadn't the time to update the picture.
The connection setup follows the following steps:
- When the server is started it calls RmxGetServiceName() to get the
name it should listen to.
- The server creates the connection, using RmxCreate(), and issues
RmxConnect() on the connection. This causes the server to block.
- When the client is started it calls RmxGetServiceName() to get the
name of the server. It obtains the name of the host from the environment
variable RMXDISPLAY (In the demo client application provided with this
article, the hostname is given on the command line.)
- The client then opens a connection using RmxOpen(). This will cause
the RmxConnect() function in the server to return.
- The server immediately issues a call to RmxRead(), which again will
cause the server to block.
- The client sends the required connection setup info to the server
using RmxWrite(). This will cause the RmxRead() call in the server to
return.
- Immediately after the call to RmxWrite(), the client calls RmxRead()
in order to get the reply from the server. This will cause the client to
block.
- If the start-up parameters are acceptable, the server starts a new
engine. The engine calls RmxGetUniqueName() to get a unique new name and
uses RmxCreate() to create the connection.
- The engine passes the engine name back to the server which, using
RmxWrite(), sends it back to the client. Meanwhile the engine calls
RmxConnect() which will cause it to block.
- The client will now return from RmxRead(). It will immediately call
RmxClose() to close the connection.
- The server calls RmxDisConnect() to close the connection instance.
- The client uses the value of the environment variable RMXDISPLAY and
the returned engine name for opening a connection to the engine using
RmxOpen().
- When the client calls RmxOpen() the engine will return from
RmxConnect().
- The client enters a loop where it calls RmxWrite() to send requests to
the engine and calls RmxRead() to receive replies. The engine will
similarly enter a loop where it calls RmxRead() to get a request from a
client, process the request and send a reply using RmxWrite().
- Although not shown in the figure, the client will eventually call
RmxClose() to close the connection. The engine will detect this, exit its
own loop, call RmxDisConnect() and RmxClose() and finally exit.
RMXPIPE.DLL
Ok, let's look at the named pipe implementation. All source is available
in RMXCOMMS.ZIP delivered with this issue of EDM/2.
Types
One thing that has to be specified when named pipes are used is the size of
the named pipe buffers. I chose to use buffers of 4096 bytes (one page).
Bigger buffers would give better throughput when big chunks of data is
being transferred - with the cost of more memory being used - but as in RMX
mostly small chunks are being sent, 4K was just about right.
const ULONG SIZE_PAGE = 4096;
const ULONG SIZE_PIPE = SIZE_PAGE;
There is a need for a buffer outside the actual named pipes as well. As I
wanted the entire data structure to reside within a (multiple if need be)
page, I defined the buffer size using the pipe size.
const ULONG SIZE_BUFFER = SIZE_PIPE - sizeof(HPIPE);
struct CONNECTION
{
HPIPE hPipe;
BYTE bytes[SIZE_BUFFER];
};
Now CONNECTION fits conveniently within a page. A user should not see this
structure, however, as it will be different depending on the underlying
communications mechanism. Therefore the only thing a user sees is a,
typedef void* HCONNECTION;
that can only be used as a handle.
ULONG RmxGetServiceName(PCSZ pcszService, ULONG* pulSize, PSZ pszName);
In the TCP/IP world there is standard way of finding server applications on
remote hosts. In the directory /etc (\TCPIP\etc on OS/2) there is a file
services that contains well-known entry points for different services.
Unfortunately there is no similar convention with named pipes.
At first I had the RMX named pipe names hard coded in RMXPIPE.DLL but while
writing this article I realised that hard coding the names effectively
would mean that nobody else could use RMXPIPE.DLL directly but would have
to modify it. Consequently I decided to implement a similar kind of
functionality as the TCP/IP \etc\services.
As there is no obvious place in OS/2 to put a file like that I chose to
require that it has to be in the same directory as RMXPIPE.DLL and that its
name has to be RMXPIPE.DAT. Currently it looks like:
#$Header$
#
# This file associates official service names with a pipe name.
#
# The form for each entry is:
# <official service name> <pipe name>
#
# RMXPIPE.DLL will automatically provide the initial required '\PIPE'
# of a name so it need not be explicitly provided.
#
rmxserver \rmx\server # RMX Server Process
rmxstarter \rmx\starter # RMX Starter Daemon
rmxtester \rmx\tester # RMX Test Server
The implementation of RmxGetServiceName() simply scans this file and
returns the corresponding pipe name.
The server uses the returned name directly with RmxCreate(). The client
cannot in general use the name directly with RmxOpen(), as the name
uniquely identifies the service within the computer but not on the network.
The name of the host is required for uniquely identifying the service on
the network. In RMX this is solved by requiring that the host name is
provided in the environment variable RMXDISPLAY. If RMXDISPLAY is
specified, RMX attempt to redirect the application to the remote host, if
it is not, the application is run locally.
[C:\]set RMXDISPLAY=\\LOKE
[C:\]start app
The client application provided along this article takes the host as a
command line argument.
ULONG RmxCreate(PCSZ pcszName, HCONNECTION* phConn);
RmxCreate() is used by a server process to create a connection. The only
argument it takes is the name of the connection and a pointer to a variable
where the connection handle will be returned. Most IPC mechanism can be
configured in a million different ways - just look at the prototype for
DosCreateNPipe() for instance - but as I only wanted to create connections
of a specific kind there was no reason to have any additional arguments in
the prototype of RmxCreate(). What RMX needs from a connection is:
- blocking behaviour, reads on an empty connection blocks
- write through, no buffering
- read-write mode, possible to read and write using the same handle
- unlimited instances, multiple instances with the same name can be created
Let's see how these requirements can be achieved with named pipes. The
prototype of DosCreateNPipe() looks like:
APIRET APIENTRY DosCreateNPipe(PCSZ pszName,
PHPIPE pHpipe,
ULONG openmode,
ULONG pipemode,
ULONG cbInbuf,
ULONG cbOutbuf,
ULONG msec);
The interesting arguments are openmode and pipemode. Openmode defines the
mode in which the pipe is opened and pipemode defines the mode of the pipe.
To get the behaviour I wanted I defined the open flags as:
const ULONG NP_OPEN_FLAGS = NP_NOWRITEBEHIND |
NP_NOINHERIT |
NP_ACCESS_DUPLEX;
and the pipe flags as:
const ULONG NP_PIPE_FLAGS = NP_WAIT |
NP_TYPE_BYTE |
NP_READMODE_BYTE |
NP_UNLIMITED_INSTANCES;
Using these constants the pipe can be created.
ULONG
rc;
HPIPE
hPipe;
rc = DosCreateNPipe(pcszName,
&hPipe,
NP_CREATE_FLAGS,
NP_PIPE_FLAGS,
SIZE_PIPE,
SIZE_PIPE,
0);
As the HCONNECTION that will be returned from RmxCreate() is in fact a
CONNECTION, some memory must also be allocated. As this takes place in a
DLL I chose to allocate the memory using DosAllocMem() directly, the main
reason being that I don't know how safe it is to call runtime library
functions inside a DLL that will be used by multithreaded application, that
are not necessarily even compiled with the same compiler.
ULONG
rc;
PVOID
pvMemory;
do
rc = DosAllocMem(&pvMemory, sizeof(CONNECTION),
PAG_READ | PAG_WRITE | PAG_COMMIT);
while (rc == ERROR_INTERRUPT);
CONNECTION *pconn = (PCONNECTION) pvMemory;
memset(pconn, sizeof(CONNECTION), 0);
pconn->hPipe = hPipe;
ULONG RmxCreateUnique(ULONG* pulSize,
PSZ pszName,
HCONNECTION* phConn);
RmxCreateUnique() is used by the engine to create a unique connection. The
caller needs to provide a buffer sufficiently large where the used name
will be returned. Internally the biggest question is how to create a
unique name. A reasonably reliable method is to use the process id, thread
ordinal and the time.
const CHAR acUniqueName[] =
"\\PIPE\\RMXPIPE\\UNIQUE\\";
PPIB
ppib;
PTIB
ptib;
DosGetInfoBlocks(&ptib, &ppib);
DATETIME dateTime;
DosGetDateTime(&dateTime);
USHORT
pid = (USHORT) ppib->pib_ulpid,
tord = (USHORT) ptib->tib_ordinal,
seconds = (USHORT) dateTime.seconds, // <= 59
hundredths = (USHORT) dateTime.hundredths; // <= 99
sprintf(pszName, "%s%hu%hu%hu%hu",
acUniqueName, pid, tord, seconds, hundredths);
and then create the named pipe in a loop that eventually either succeeds
properly or fails completely.
do
{
GetUniqueName(achUniqueName);
rc = RmxCreate(achUniqueName, phConn);
}
while (rc == RMXERR_CONNECTION_BUSY);
ULONG RmxOpen(PCSZ pcszHost, PCSZ pcszPort, HCONNECTION* hConn);
RmxOpen() is used by a client process to open a connection. The arguments
it takes are the name of the host and the port to open and a pointer to a
variable where the connection handle will be returned. When named pipes
are used the host name will be the name of a server and the port the name
of a named pipe.
To get the qualified name we simply need to concatenate the host and the
port names. In order to do that we need a buffer. Instead of first
allocating a buffer for this purpose and then another for the actual
connection handle we first allocate the connection handle buffer and use it
for storing the string. The allocation is done pretty much the same way as
in RmxCreate().
CONNECTION
*pconn = (PCONNECTION) pvMemory;
PSZ
pszName = (PSZ) pconn;
pszName[0] = 0;
if (pcszHost)
strcpy(pszName, pcszHost);
strcat(pszName, pcszPort);
The host name can be NULL which effectively means that we attempt to open
the pipe on the same computer. Once the name is built we can open the
actual pipe.
In RMX this function will be called behind the scenes of an application
which may be doing a lot of things, e.g. open a large number of files.
Hence we make sure the function does not fail because there are no handles
left.
const ULONG NP_MODE_FLAGS = OPEN_FLAGS_WRITE_THROUGH |
OPEN_FLAGS_FAIL_ON_ERROR |
OPEN_FLAGS_NO_CACHE |
OPEN_FLAGS_SEQUENTIAL |
OPEN_SHARE_DENYNONE |
OPEN_ACCESS_READWRITE |
OPEN_FLAGS_NOINHERIT;
HPIPE
hPipe;
ULONG
rc;
do
{
ULONG
ulActionTaken; // Not used.
rc = DosOpen(pszName,
&hPipe,
&ulActionTaken,
0L,
FILE_NORMAL,
FILE_OPEN,
NP_MODE_FLAGS,
0L);
if (rc == ERROR_TOO_MANY_OPEN_FILES)
{
LONG
lReqCount = 1; // We need one additional handle
ULONG
ulCurMaxFH; // Not used.
DosSetRelMaxFH(&lReqCount, &ulCurMaxFH);
}
}
while (rc == ERROR_TOO_MANY_OPEN_FILES);
Then we set the state of the pipe the same way as it was set in
RmxCreate(). I am not entirely sure whether this is necessary or if the
state is automatically set to the same as what was used when the pipe was
created, but better safe than sorry.
if (rc == NO_ERROR)
{
// The connection must be blocking.
rc = DosSetNPHState(hPipe, NP_WAIT |
NP_READMODE_BYTE);
...
ULONG RmxClose(HCONNECTION hConn);
RmxClose() is simply implemented by deallocating the memory that was
allocated in RmxCreate(), and closing the pipe
CONNECTION
*pconn = (PCONNECTION) hConn;
DosClose(pconn->hPipe);
DosFreeMem(pconn);
ULONG RmxConnect(HCONNECTION hConn);
This function is mapped directly to DosConnect(). It blocks until a client
opens the pipe.
CONNECTION
*pconn = (CONNECTION*) hConn;
ULONG
rc = 0;
do
rc = DosConnectNPipe(pconn->hPipe);
while (rc == ERROR_INTERRUPT);
ULONG RmxDisConnect(HCONNECTION hConn);
Like RmxConnect() is mapped to DosConnect(), RmxDisConnect() is mapped to
DosDisConnect().
CONNECTION
*pconn = (CONNECTION*) hConn;
DosDisConnectNPipe(pconn->hPipe);
ULONG RmxWrite(HCONNECTION hConn, PCBYTE pbBuffer,
ULONG ulBytesToWrite);
Okay, now we are getting to the more interesting stuff. This functions
takes a handle to a connection, a pointer to a buffer and an unsigned long
specifying how many bytes to write. Contrary to usual write-functions, it
does not return the number of bytes written. This is because in RMX, where
this function is used for implementing RPCs, the function must either
succeed or completely fail. Also, if the number of bytes to transfer is
too large for the underlying implementation (in this case named pipes) the
function must itself deal with the task of splitting the message into
smaller packets.
CONNECTION
*pconn = (PCONNECTION) hConn;
BYTE
*pbData = pconn->bytes;
The length of the entire message is sent in the first 4 bytes as an
unsigned integer.
*((ULONG*) pbData) = ulBytesToWrite;
ULONG
ulBytesInBuffer = SIZE_BUFFER - sizeof(ULONG);
ULONG
ulBytesToCopy = ulBytesInBuffer < ulBytesToWrite ?
ulBytesInBuffer : ulBytesToWrite;
memcpy(pbData + sizeof(ULONG), pbBuffer, ulBytesToCopy);
Okay, so what is happening here? After the size, we copy as much actual
data as can fit into the data area of the connection handle. An
alternative would be to first send the size, and then send the data
directly out of the buffer provided by the caller. But that would mean
that no matter how small a message, it would always be sent in at least two
separate packets. In the code that follows note that during the first
DosWrite(), pbData points to the connection buffer, but on subsequent calls
it points directly to the buffer given by the caller.
ULONG
ulBytesLeft = ulBytesToWrite + sizeof(ULONG),
ulBytesToSend = ulBytesToCopy + sizeof(ULONG);
LONG
lBytesSent = -sizeof(ULONG);
ULONG
rc = RMXAPI_OK;
do
{
ULONG
ulBytesSent;
rc = DosWrite(hPipe, pbData, ulBytesToSend, &ulBytesSent);
lBytesSent += ulBytesToSend;
ulBytesLeft -= ulBytesToSend;
if (ulBytesLeft != 0)
{
pbData = (PSZ)pbBuffer + lBytesSent;
ulBytesToSend = SIZE_PIPE < ulBytesLeft ?
SIZE_PIPE : ulBytesLeft;
}
}
while (ulBytesLeft != 0);
The funny looking plus and minus sizeof(ULONG) is there because the first
message contains the size information which is not part of the actual
message and must be excluded when we figure out whether data needs to be
copied still. This ought to be cleaned up somwhat, I get nervous when
signed and unsigned quantities are mixed in this fashion.
ULONG RmxRead(HCONNECTION hConn, PBYTE pbBuffer, ULONG ulSize,
ULONG* pulBytesRead);
This function takes a handle to a connection, a pointer to a buffer, an
unsigned long that specifies the size of the buffer, and a pointer to an
unsigned long where the size of the read message is returned. If the
buffer is not big enough the function returns an error code indicating that
the buffer must be enlarged and the required size is returned in the
variable pointed to by pulBytesRead.
If the variable indicating the length is 0, this is a true new call to
RmxRead().
CONNECTION
*pconn = (PCONNECTION) hConn;
HPIPE
hPipe = pconn->hPipe;
ULONG
*pulLength = (ULONG*) pconn->bytes;
If it is 0, we read the size as a separate operation. We have to do that
because we do not want to read to the size to the buffer provided by the
caller, but to the buffer associated with the handle.
if (*pulLength == 0)
{
ULONG
ulBytesLeft = sizeof(ULONG),
ulBytesRead = 0;
do
{
PBYTE
pbData = ((PBYTE) pulLength) + ulBytesRead;
DosRead(hPipe, pbData, ulBytesLeft, &ulBytesRead);
ulBytesLeft -= ulBytesRead;
}
while (ulBytesLeft != 0);
if (*pulLength > ulSize)
{
*pulBytesRead = *pulLength;
return RMXERR_ENLARGE_BUFFER;
}
}
Perhaps it is overkill to read 4 bytes in a loop like that, but at least
we'll be certain that we get the data we are interested in. The loop for
reading the actual message data looks pretty much the same. Note that
pbData now points to the buffer provided by the caller.
ULONG
ulBytesLeft = *pulLength,
ulBytesRead = 0,
ulTotalBytesRead = 0;
do
{
PBYTE
pbData = pbBuffer + ulTotalBytesRead;
rc = DosRead(hPipe, pbData, ulBytesLeft, &ulBytesRead);
ulBytesLeft -= ulBytesRead;
ulTotalBytesRead += ulBytesRead;
}
while (ulBytesLeft != 0);
*pulBytesRead = *pulLength;
The length is set to 0 to indicate that the entire message has been read.
*pulLength = 0;
RMXTCPIP.DLL
Okay, now we begin to look at the equivalent TCP/IP implementation. When I
initially specified the communications API I followed the named pipe model
fairly closely. The reason was that I wanted it to be easy to implement
the RMX API on top of them, but also because I had very little experience
with TCP/IP and I didn't exactly know how they work. But, I was confident
that it would later be possible to fit them into the RMX model.
If we look at named pipes and TCP/IP in a multithreaded client-server
situation, the basic scenario can be pictured as (_beginthread() is called
when a client opens the connection the server blocks on):
From this figure it is fairly obvious that there is not a direct mapping
between named pipes and sockets. And while it is trivial to implement the
RMX API using named pipes, sockets require a little bit more thought.
The main problem is the difference in how multiple connections with the
same name is handled. With named pipes the server simply calls
DosCreateNPipe() and DosConnect() multiple times. The name given to
DosCreateNPipe() is each time the same. With sockets you first create a
socket using socket(), bind() and listen(), and then you call accept()
multiple times. Each call to accept() returns a new socket handle. Like
most other things in computer science this can be solved by adding an extra
level of indirection.
ULONG RmxGetServiceName(PCSZ pcszService, ULONG* pulSize, PSZ pszName);
Now that we are running on top of TCP/IP we can directly use the
\tcpip\etc\services file. The file should contain entries like:
rmxserver 4711/tcp
rmxstarter 4712/tcp
rmxsecurity 4713/tcp
rmxtester 4714/tcp
The first column contains the public name of a server, and the second the
port number and the protocol to be used. The port numbers are something
that basically the world has to agree upon, so I just chose a few numbers
that did not seem to be used by anybody. They are easily changed if they
happen be in conflict with something else. The socket library directly
provides functions for dealing with this file. As the port is returned in
network format we have to convert it to host format using ntohs().
servent
*s = getservbyname((PSZ) pcszService, "tcp");
if (!s)
return RMXERR_UNKNOWN_SERVICE;
sprintf(pszName, "%d", ntohs((unsigned short) s->s_port));
ULONG RmxCreate(PCSZ pcszName, HCONNECTION* phConn);
RmxCreate() is used by a server process to create a connection. And as it
is supposed to be called several times with the same name we cannot
directly map it onto the usual socket creation functions. The first thing
we do is to convert the given name to a port number (the user supposedly
has called RmxGetServiceName() before this).
int
port = atoi(pcszPort);
return CreateConnection(htons((unsigned short) port), phConn);
All work is done by CreateConnection() which is also used by
RmxCreateUnique(). As we need to be able to detect whether the caller
actually wants to create a new socket or if he only wants to have a new
instance of the same socket we need to have an additional data structure
for that purpose.
struct SOCKET
{
unsigned short port; // The port number in network format.
int socket; // The socket handle.
unsigned users; //Number of users of this instance.
SOCKET* next;// Pointer to next instance (or NULL).
SOCKET* prev;// Pointer to previous instance (or NULL).
};
const ULONG SIZE_BUFFER = SIZE_CONNECTION -
sizeof(SOCKET*) -
sizeof(int);
struct CONNECTION
{
SOCKET* sharedSocket;
int privateSocket;
BYTE bytes[SIZE_BUFFER];
};
The first thng CreateConnection() does is to get an instance of SOCKET. I
don't show it here (it's all available in the source code) but PickSOCKET()
scans through all existing SOCKET instances and if it finds one instance
where the port number is the same as the one requested, it increases the
use count and returns it. Otherwise it creates a new instance.
SOCKET
*pSocket = PickSOCKET(port);
if (pSocket->socket == 0)
rc = InitializeSOCKET(pSocket);
PCONNECTION
pconn = AllocConnection();
pconn->sharedSocket = pSocket;
*phConn = pconn;
If it was an all new instance there is no real socket associated with it
yet. The socket is created in InitializeSOCKET() that as its first action
calls CreateSocket(). CreateSocket() creates the socket handle and deals
with system call interruptions. It is in a separate function because it is
also used by RmxOpen().
static int CreateSocket()
{
int
serrno = 0,
socket = 0;
do
{
socket = ::socket(AF_INET, SOCK_STREAM, 0);
if (socket < 0)
serrno = sock_errno();
}
while (serrno == SOCEINTR);
if (serrno != 0)
return 0;
return socket;
}
The socket handle is just a handle, it cannot be used for anything yet. It
must first be bound which means that a local address is assigned to it. If
the specified port is 0, the system allocates a unique port on behalf of
the caller.
int
socket = CreateSocket();
sockaddr_in
in;
in.sin_family = AF_INET;
in.sin_port = pSocket->port;
in.sin_addr.s_addr = INADDR_ANY;
::bind(socket, (sockaddr*) &in,
sizeof(sockaddr_in));
The operation is not complete yet. We need to have a connection request
queue for the incoming requests. That is taken care of by listen().
::listen(socket, SOMAXCONN);
If the socket was created without an explicit port number, we ask the
system what the actual port number is.
if (pSocket->port == 0)
{
int
size = sizeof(sockaddr_in);
getsockname(socket, (sockaddr*) &in, &size);
pSocket->port = in.sin_port; // Note, network
order
}
pSocket->socket = socket;
The call to AllocConnection() simply allocates an instance of CONNECTION
and returns it. This completes the implemetation of RmxCreate(). Although
not necessarily entirely obvious this arrangement allows RmxCreate() to be
called multiple times using the same port number, yet the actual socket is
created only once, on subsequent calls only its use-count is increased.
ULONG RmxCreateUnique(ULONG* pulSize,
PSZ pszName,
HCONNECTION* phConn);
RmxCreateUnique() is implemented exactly like RmxCreate() except that the
port number provided is zero. That will then cause the system to allocate
some unique port.
CreateConnection(0, phConn);
PCONNECTION
pconn = (PCONNECTION) *phConn;
sprintf(pszPort, "%d", ntohs(pconn->sharedSocket->port));
ULONG RmxOpen(PCSZ pcszHost, PCSZ pcszPort, HCONNECTION* hConn);
As the host and port names are expressed textually they must be converted
into integers before we can use them. The port is simply converted using
atoi(), the host needs a little bit more attention.
int
port = atoi(pcszPort);
First we check whether the host has been specified in ordinary dotted
decimal notation (e.g. 192.26.110.20).
unsigned long
address = 0;
if (pcszHost)
{
address = inet_addr((PSZ) pcszHost);
if (address == (unsigned long) -1)
{
If it hasn't we attempt to look it up from the hosts file, or from a
nameserver if present (handled automatically by gethostbyname()).
hostent
*host = gethostbyname((PSZ) pcszHost);
address = *(unsigned long*) host->h_addr;
}
}
When we have the complete address, we can create and connect the socket.
First we need the actual socket handle.
int
socket = CreateSocket();
Just like when we were creating the connection this socket cannot be used
for anything. It has to be connected to a real address. We use the port
and address we just figured out. Again the port must be converted into
network format. The address is in network format already when it is
returned by inet_addr() or gethostbyname().
sockaddr_in
sin;
sin.sin_family = AF_INET;
sin.sin_port = htons((unsigned short) port);
sin.sin_addr.s_addr = address;
::connect(socket, (sockaddr*) &sin,
sizeof(sockaddr_in));
The only thing left is to allocate the connection handle.
PCONNECTION
pconn = AllocConnection();
pconn->privateSocket = socket;
*phConn = pconn;
As the memory returned by AllocConnection() is set to 0,
pconn->sharedSocket will also be NULL.
ULONG RmxClose(HCONNECTION hConn);
RmxClose() is used by both the client and the server to close a connection.
The client will never have a shared socket, and the server should not -
provided it has first (as it ought to) called RmxDisConnect() - have a
private socket.
PCONNECTION
pconn = (PCONNECTION) hConn;
if (pconn->sharedSocket)
ReleaseSOCKET(pconn->sharedSocket);
int
socket = pconn->privateSocket;
DosFreeMem(pconn);
if (socket)
::soclose(socket);
ReleaseSOCKET() decrements the use count of the shared socket, and if it
reaches 0, it closes the real socket and deletes the SOCKET instance.
ULONG RmxConnect(HCONNECTION hConn);
This function can be directly mapped to the socket call accept(). Accept()
blocks the caller until a client opens a connection.
PCONNECTION
pconn = (PCONNECTION) hConn;
int
socket = ::accept(pconn->sharedSocket->socket, 0, 0);
pconn->privateSocket = socket;
Accept() returns a unique socket handle that can be used for communicating
with the client that opened the connection
ULONG RmxDisConnect(HCONNECTION hConn);
RmxDisConnect() is called by the server to close a specific client
connection.
PCONNECTION
pconn = (PCONNECTION) hConn;
::soclose(pconn->privateSocket);
pconn->privateSocket = 0;
The shared socket is still there, so we can use the same connection handle
for servicing a new client by calling RmxConnect().
ULONG RmxWrite
(HCONNECTION hConn, PCBYTE pbBuffer, ULONG ulBytesToWrite);
ULONG RmxRead
(HCONNECTION hConn, PBYTE pbBuffer, ULONG ulSize, ULONG* pulBytesRead);
The socket implementation of RmxRead() and RmxWrite() is almost identical
to the named pipe implementation. The main difference is that instead of
using DosRead() and DosWrite() we use recv() and send().
RMXCOMMS.DLL
If you look at the makefiles of RMXPIPE.DLL and RMXTCPIP.DLL you'll see
that I generate no import library from the DLLs. That is on purpose. The
way I see it, these two DLLS are only an implementation of an API. Any
program that uses the API should be able to use another DLL that implements
the same API using another communications mechanism.
If an application is linked with an import library generated from
RMXPIPE.DLL, for example, then that application is forever married to the
DLL. If the application should use another DLL it must be relinked with an
import library generated from the other DLL.
If the application is instead linked with RMXCOMMS.LIB which is generated
from RMXCOMMS.DLL, the application need never be relinked in order to use a
particular implementation of the RMX communications API.
RMXCOMMS.DLL provides exactly the same interface as RMXPIPE.DLL and
RMXTCPIP.DLL, and is in fact built using the same header as they are. Its
implementation is, however, quite different.
First a few types are defined: PPFN is a pointer to a function pointer and
Function is structure holding an ordinal and a pointer to a function
pointer.
typedef PFN* PPFN;
struct Function
{
ULONG ulOrdinal;
PPFN ppfnAddress;
};
Then a number of module variables. As you notice there is one function
pointer for each function defined by the RMX communications API. The
addresses of the function pointers are stored, together with the
corresponding ordinal, in an array of function structures.
static HMODULE hmodRmxComms;
static ULONG RMXENTRY (*rmxClose)(HCONNECTION);
static ULONG RMXENTRY (*rmxConnect)(HCONNECTION);
static ULONG RMXENTRY (*rmxCreate)(PCSZ, HCONNECTION*);
static ULONG RMXENTRY (*rmxCreateUnique)(ULONG*, PSZ, HCONNECTION*);
static ULONG RMXENTRY (*rmxDisConnect)(HCONNECTION);
static ULONG RMXENTRY (*rmxGetServiceName)(PCSZ, ULONG*, PSZ);
static ULONG RMXENTRY (*rmxOpen)(PCSZ, PCSZ, HCONNECTION*);
static ULONG RMXENTRY (*rmxRead)(HCONNECTION, PBYTE, ULONG, ULONG*);
static ULONG RMXENTRY (*rmxWrite)(HCONNECTION, PCBYTE, ULONG);
static Function afFunctions[] =
{
{ ORD_RMXCLOSE, (PPFN) &rmxClose },
{ ORD_RMXCONNECT, (PPFN) &rmxConnect },
{ ORD_RMXCREATE, (PPFN) &rmxCreate },
{ ORD_RMXCREATEUNIQUE, (PPFN) &rmxCreateUnique },
{ ORD_RMXDISCONNECT, (PPFN) &rmxDisConnect },
{ ORD_RMXGETSERVICENAME, (PPFN) &rmxGetServiceName },
{ ORD_RMXOPEN, (PPFN) &rmxOpen },
{ ORD_RMXREAD, (PPFN) &rmxRead },
{ ORD_RMXWRITE, (PPFN) &rmxWrite }
};
const ULONG cFunctions = sizeof(afFunctions)/sizeof(Function);
Let's then look at the DLL initialization routine of RMXCOMMS.DLL. The
first thing is to query the value of the environment variable RMXCOMMS.
PCSZ
pcszRmxComms;
if (DosScanEnv("RMXCOMMS", &pcszRmxComms) != NO_ERROR)
return 0;
The value of RMXCOMMS is assumed to be the name of the actual
communications DLL to use.
if (DosLoadModule(0, 0, pcszRmxComms, &hmodRmxComms) != NO_ERROR)
return 0;
Finally the functions can be resolved.
for (int i = 0; i < cFunctions; i++)
{
ULONG
ulOrdinal = afFunctions[i].ulOrdinal;
PPFN
ppfnAddress = afFunctions[i].ppfnAddress;
ULONG
rc = DosQueryProcAddr(hmodRmxComms, ulOrdinal, 0, ppfnAddress);
if (rc != NO_ERROR)
return 0;
}
This means in practice that once the DLL has successfully been loaded, the
function pointers will point at valid functions. Now each exported
function can simply be implemented in the following fashion.
ULONG RMXENTRY RmxCreate(PCSZ pszName, HCONNECTION* phConn)
{
return rmxCreate(pszName, phConn);
}
Example
Suppose we have an application CLIENT.EXE that has been linked with
RMXCOMMS.LIB which was generated from RMXCOMMS.DLL. And suppose that there
exists the proper communications DLLs RMXPIPE.DLL, RMXTCPIP.DLL and
RMXIPX.DLL. Then the following three client processes will all use
different DLLS.
[C:\]set RMXCOMMS=RMXPIPE
[C:\]start client.exe
[C:\]set RMXCOMMS=RMXTCPIP
[C:\]start client.exe
[C:\]set RMXCOMMS=RMXIPX
[C:\]start client.exe
Sample Programs
Along with this issue of EDM/2 you should get RMXCOMMS.ZIP that contains
RMXCOMMS.DLL, RMXPIPE.DLL and RMXTCPIP.DLL, and the test/demo programs
SERVER.EXE, ENGINE.EXE and CLIENT.EXE, along with all the source. In order
to try them out, you should place the DLLs in some directory along the
LIBPATH. Remember to place the file RMXPIPE.DAT in the same directory as
RMXPIPE.DLL, and remember to update the \tcpip\etc\services file if you
intend to use RMXTCPIP.DLL. SERVER.EXE and ENGINE.EXE should be in the
same directory.
If you want to run the server/engine pair and the client in separate
computers you'll need a network. The DLLs must of course be present in
both computers.
Starting the server
Launch a windowed command prompt and go to the directory where SERVER.EXE
resides.
[C:\RMX\DEMO]set RMXCOMMS=RMXPIPE
[C:\RMX\DEMO]server
The server prints a notification message when it starts.
Starting the client
The client requires a few command line parameters in order to start. If it
is started with the wrong parameters it prints:
usage: client [-t #] ([-h host]|-e engine) [-v]
-t # : Number of threads
-h host : The name of the host
-e engine : Use this as engine connection
-v verbose: More chit chat
Note the blank between the flag and the value
example: client -t 5 -h \\ODIN
client -v
The -t flags specifies how many threads the client should use. As default
it uses 5. The flag -h specifies the host and -v turns on verbose mode
(recommended). If you are running locally don't specify a host,
[C:\RMX\DEMO]set RMXCOMMS=RMXPIPE
[C:\RMX\DEMO]client -v
otherwise the argument of the flag should be the name of the computer where
the server is running. The host name varies of course depending on which
communications DLL you are using. In my environment I use:
[C:\RMX\DEMO]set RMXCOMMS=RMXPIPE
[C:\RMX\DEMO]client -h \\loke -v
with named pipes and
[C:\RMX\DEMO]set RMXCOMMS=RMXTCPIP
[C:\RMX\DMEO]client -h loke -v
or
[C:\RMX\DEMO]set RMXCOMMS=RMXTCPIP
[C:\RMX\DMEO]client -h 192.26.110.21 -v
with TCP/IP.
When started the client opens a connection to the server and asks it to
start an engine, the server starts the engine that creates a connection
whose name is transferred back to the client. The client opens a new
connection to the engine and starts sending messages of varying size to the
engine that subsequently sends them unaltered back. The client then
verifies that there has been no changes in the message.
Compiling the source
I've written the code using Borland C++ 1.5 so if you have that compiler
you should be able - perhaps with a little tinkering - to build the DLLs
and the applications. If you have another compiler you'll probably have to
make more modifications. If you do make changes I'd appreciate if you
would mail them to me so I can include them in the "official" version.
A word of caution
I've been using the RMXPIPE.DLL implementation for quite some time while
I've been developing RMX and I have not experienced any problems. However,
now when I tested the programs for this article, the client occasionally
received an error code (ERROR_REQ_NOT_ACCEP to be exact) when it tried to
send data to the server. I havn't experienced any problems at all with
RMXTCPIP.DLL so I blame the named pipe implementation of OS/2.
Implementing RPCs
Even if this API hides quite a few of the complex details of client-server
programming, it is still quite tedious to work with. Fortunately it can be
made easier. This article is long as it is so I won't go into any details
but I'll show you how WinInitialize is implemented in RMX. I might return
to this subject in a later article. C++ templates can make the world a
better place to program in...
Client
HAB APIENTRY WinInitialize(ULONG flOptions)
{
ClientConnection
&c = ClientConnection::connection();
Packet
&p = c.freshPacket();
p << RMX_OS2PMWIN;
p << (ORDINAL) ORD_WIN32INITIALIZE;
p << flOptions;
c.sendReceive();
HAB
hab;
p >> hab;
return hab;
}
Server
void PMWin::winInitialize(Packet& p)
{
ULONG
flOptions;
p >> flOptions;
HAB
hab = WinInitialize(flOptions);
p.reply() << hab;
}
Conclusion
So, that was the low level transfer mechanism of RMX. Feel free to use it
in any way you see fit and don't hesitate to mail me if there is something
you wonder about. The next article will be about how to mark an
application for use with RMX.
