Borland Delphi Informant Magazine Home Click HERE...FREE RoboHELP Starter Kit


Member Services
FREE Trial Issue
New Subscription
Renew Subscription
Delphi CD-ROM
Report Problems
Change of Address


Delphi Informant
Features
Case Studies
News
New Products
Book Reviews
Product Reviews
Opinion
Back Issues

Downloads
Article files
Third-party files
Upload A File

Informant
ICG News
Contact Us
Advertise With Us
Write For Us
Delphi PowerTools Web Site
 
Delphi PowerTools Web Site
 •

A Multimedia Assembly Line: Part II


 •

Control Panel Applets


 •

Run-time ActiveX


 •

Dropping Hints: Part II


 •

Multiple Inheritance: In Delphi?





Tell a friend
about this article!




Distributed Delphi

Multi-threading / DCOM / Database / Delphi 4, 5

Request Threads

One Solution to Resource-sharing Problems

In an ideal world, a server system would have unlimited resources available to an unlimited number of clients. That world doesn't exist just yet. For example, a database server can only support a limited number of client connections. The database, in this case, is the limiting resource.

One way around this problem is the use of Transaction Processing Monitors, e.g. Microsoft's Transaction Server (MTS), or BEA's Tuxedo. This provides for three-tier distributed processing, with the TP Monitor taking care of resource sharing and other important things, such as transactions and data integrity. Indeed, Delphi 4/5 allows for easy creation of MTS objects, but MTS has its drawbacks, and is not the answer for all situations. Other TP Monitors may be overkill for a small system, or simply too expensive in licensing, development, and support costs. There's also a steep learning curve to consider.

This article describes a resource-sharing method implemented using Delphi 4 Client/Server Suite and Microsoft's Distributed Component Object Model (DCOM), with the resource being a database (the techniques apply equally well to Delphi 5). The method could equally be applied to any modern programming language and operating system that supports threads and remote procedure calls. The resource need not be a database; it could be a file server, printer, etc. The method described in this article is based on one used within a workflow-imaging trade finance product called Prexim, developed by China Systems.

One-to-one Connections

It's a simple task to create a multi-threaded DCOM server using Delphi 4 Client/Server Suite. Such a server is capable of providing services to a potentially unlimited number of clients, with the basic limiting factors being memory and processing power.

Servers are never that simple, however. For example, they usually rely on services provided by other servers, such as a relational database server. Connections to these underlying servers are generally expensive in terms of resource usage, and must, therefore, be used sparingly. It isn't practical for each thread of a server to own a dedicated connection to another database server. For example, if there are 100 clients connected to a server, there will be 100 corresponding connections to the underlying database server (see Figure 1).


Figure 1: One-to-one connections.

This is clearly impractical, and in many cases impossible. It's a huge waste of resources, as those database connections are unlikely to be used all the time by all the clients. A better solution would be for clients to share these precious resources.

Threads

How do you share a connection to a database server? A simple DCOM server would create a dedicated connection to the database server for each client connection. What's needed is a pool of connections that all clients can share. This sharing must be transparent to the clients who need not be aware of such complexities of implementation.

One solution is for the DCOM server to open a number of connections to the database server when it starts. The number of connections may be configurable and fixed once the DCOM server starts. Preferably, it should be dynamic and increase or decrease on the fly depending upon the load, i.e. load balancing.

Now when a client connects to the DCOM server, it doesn't create a new connection to the database server. Instead it asks one of the threads created earlier to do the processing and return the results, which in turn will be passed back to the client (see Figure 2).


Figure 2: Request thread connections.

There are two types of threads being discussed here. When a client connects to a multi-threaded DCOM server, the server spawns a thread. This thread, which we'll call a "client thread," is responsible for servicing that client. Therefore, each client connection to a DCOM server has its own client thread. We'll call the threads that are spawned when the server initializes and created before any clients connect to the server "request threads." It's these limited request threads that are shared by the potentially unlimited client threads.

The problem now is how to communicate with those request threads. We need an efficient method of inter-thread communication.

Communication

To get straight to the point, one answer is to use good-old-fashioned First-In-First-Out (FIFO) queues. This avoids starvation, because no priorities are involved. Because we're dealing with threads, synchronization of access to these queues is critical to avoid difficult-to-find data corruption bugs. Using mutex objects, semaphores, or many of the other objects available in modern operating systems can solve this. Luckily for us, Delphi provides a simple interface to these potentially difficult-to-use operating system objects.

Communication between these threads works in the following order. The DCOM server starts and spawns one or more request threads. Each request thread has its own connection to the database server, and when a client connects to the DCOM server, a client thread is spawned. The client thread can't talk directly to the database server because it has no connection to it. For the client to talk to the database server, it must ask a request thread to perform that task on its behalf. The client thread does this by creating an inbound request record, and placing it onto the inbound request queue.

A request record describes what processing the client thread wants the request thread to perform. The client now waits for the result (as we will discuss later, it need not block on the wait). One of the request threads takes the item out of the inbound queue and performs the task, which involves talking to the database server via the connection the request thread owns, e.g. the BDE.

The results are placed into an outbound request record and put onto the outbound request queue. An outbound request record contains the results of the processing, and any other relevant information, such as error messages. The client thread, which has been waiting for the result, takes the outbound request record off the outbound queue. After any further processing, the results are returned to the calling client (see Figure 3).


Figure 3: A diagram of the communication between the client and request threads.

Spawning Request Threads

When the DCOM server starts, it must spawn one or more request threads for the connections to the database server. More threads give more throughput, but increase memory use. Also, at some point the throughput from having more threads decreases. Ideally, some method of load balancing should be used, but that's beyond the scope of this article.

The reqint.pas unit contains a function named RequestThreadsStart that must be called by the DCOM server when it starts. This function initializes synchronization objects and spawns the required number of request threads. It allows for initialization failure of one or more request threads, i.e. five requests may be asked for, but, perhaps, only two request threads can actually be created. The minimum number required is specified as an argument to the function.

The first thing RequestThreadsStart does is create the inbound and outbound queues, and the semaphores used to coordinate reading from and writing to those queues. It would be very inefficient to poll the queues waiting for new records to arrive, so a more efficient method using semaphores is used instead.

Making Requests

The client threads must create a request record detailing what is required of the request threads. This record must then be placed onto the inbound queue. Method1 and Method2 demonstrate how simple this is (see Figure 4). An inbound request record is filled in with the request details. Then g_RequestInterface.RequestSync is called to send the request to the request threads, and wait for the result.


Figure 4: The example DCOM server displayed in the Type Library editor.

The RequestSync function is an interface for RequestIn and RequestOut, where RequestIn passes the request onto the request threads, and RequestOut checks to see if the results are available for collection. If Method1 or Method2 wanted to make an asynchronous request, they would call RequestIn and RequestOut separately without calling upon RequestSync.

How does a client thread know which results to collect from the outbound queue? By taking results with the correct result handle. When a request is made, it's given a unique result handle that is simply a unique number (unique for each server process), the current date and time, and a random string of characters. The date and time are stored in the result handle, so the server knows if a result has timed out. A random string of characters is placed on the end to reduce the chance that another client will try to steal another's results. Remember that the result handle may be used asynchronously at a later point by the client, so it makes sense to make it a single string instead of multiple strings.

As explained earlier, semaphores are used to signal both client threads that results are available for collection, and request threads that a request has been made. g_SemInQ is the inbound semaphore, and g_semOutQ is the outbound semaphore.

Processing Requests

TRequestThread.Execute contains a simple loop that shows what each request thread does. First it takes the first request record it can get from the inbound queue (via the function TakeFromQueue). Next it processes the item, places the results onto the outbound queue, and frees the memory used by the inbound request, as it's no longer required (see Figure 5).

// Take out of g_InQ and place items in g_OutQ.

while not Self.Terminated do begin

  inq := TakeFromQueue;

   if Self.Terminated then

    Break;

   // Process it.

  outq := ProcessItem(inq);

   if outq = nil then

    Continue;

   // Add it to the out Q.

  AddToQueue(inq^.request_in.rhandle, outq);

  Dispose(inq);

end;

Figure 5: TRequestThread.Execute showing what each request thread does.

Collecting Results

The TRequestThread.RequestOut member function is used to collect the results from the request threads. Supplied with a request handle (previously created when the client thread made a request via RequestIn) and a place to store the results, it first checks to see if the request has timed out. If a request is over a certain age, that request's results can't be collected because they will have been garbage collected and removed from the outbound queue. If the results weren't purged, the outbound queue would eventually grow too large if the client threads didn't collect their results. Remember that if an asynchronous request had been made, the client may be on another machine. The client program may exit before collecting the results, or the network connection may go down. There are many reasons why results may never get collected. Deciding if a result may not get collected isn't possible.

Having resolved that the request isn't too old, the outbound queue semaphore (g_SemOutQ) is checked. If it's signaled, then there are results in the queue, else the queue is empty and hence the results aren't available yet. The queue is searched for a matching request that hasn't been handled, i.e. another thread hasn't already taken the results. Because the list is locked, when it's searched there is no need for mutexes or other thread synchronization methods. Only one request thread can search the outbound queue at a time. A result can't be taken from the queue during the search, so its handled state is flagged as being True.

If the result was found, it can then be removed from the queue and returned to the client thread. When no result is found, the outbound queue semaphore must be re-signaled to indicate that there are still pending results. Without re-signaling, other request threads would assume the queue is empty.

Asynchronous

Imagine an extension to Delphi that allowed asynchronous function calls without the programmer needing to worry about how it's actually implemented. Figure 6 shows that the asynchronous function can be called as per any other function with no special considerations.

function AsyncExample(const in_arg: Integer;

   var out_arg: string): Boolean; Async;

begin

   // Do some processing. Set the value of out_arg.

   // Return some result.

  Result := True;

end;

 

procedure UsingTheExample;

var

  AsyncReturn: Boolean;

  SomeString: string;

begin

   // Call our async function.

  AsyncReturn := AsyncExample(123, SomeString);

 

   // At this point we can continue doing something

   // else while that function does whatever it does.

 

   // Let's get the result.

   if AsyncReturn then

    AnotherString:= 'The string result is:' + SomeString

  else

    AnotherString:= 'Error';

end;

Figure 6: Calling an asynchronous function with no change in method.

Control returns to the procedure before the function has completed and returned its result. The procedure would only block waiting for the result if it tried to use one of the values returned, or set by the asynchronous function, e.g. AsyncReturn or out_arg. Wouldn't that be a very useful extension to any language?

With request threads, it's possible to do this now within a process and outside of the process. For example, asynchronous COM calls are possible with request threads. The client would call the COM method as usual, but instead of getting its result, the client would get a unique handle. At a later point, the client would call the COM method again with the handle supplied earlier and would now get the results back. The client who originally made the call need not supply the handle. Another client process on another machine could collect the results as long as it used the correct handle.

Unfortunately, the client must be aware that the call is asynchronous, and must store the handle. There is also the possibility that the client will never collect the result, so some garbage collection must be performed periodically within the server. Security must also be taken into consideration so that clients don't try to steal results meant for others.

Improvements

There are many ways to improve upon the simple method shown in this article, and there are still many more things that haven't been described.

First, it must be said that the example code isn't what it should be and can clearly be improved upon. Delphi is an object-orientated language, but the example code given doesn't make much use of its object-orientation. A generic request threads class would be a very powerful class indeed.

Request threads can span across multiple calls. For example, a client thread may need to make multiple calls to the same request thread because that request thread has state, e.g. a cursor connection to the database server, or perhaps even an open database transaction. This is possible within request threads simply by locking a request thread to one client thread via a unique handle. This handle can be passed each time the client thread needs to use a specific thread. Locking a thread doesn't necessarily mean other client threads can't use it while it's idle. For example, if the request thread is locked because it has a cursor open, it doesn't mean other client threads can't use it to read other tables as long as the cursor isn't affected. It all gets complicated and other issues arise, such as unlocking threads, orphaned request threads, etc. For that reason, it's not described in this article.

Because requests are defined within records and are used within queues, it allows for easy recording of requests for later playback. For example, a database transaction could be performed by recording requests and storing them in a list instead of sending them to request threads one by one. Later, the entire list could be sent to a request thread for immediate processing as a single block of requests. The block of requests could be processed within one database transaction within the request thread.

The example code also doesn't show how to ensure that the DCOM server has completed initialization before client calls are accepted. If a client call is received before the queues have been set up, for example, then problems may occur. Request records may need to contain a large amount of data, as they need to support all types of requests. To reduce memory usage, variant records should be used, or perhaps even variant variables.

Request threads beg for tuning parameters, e.g. TTL (time-to-live) values for request in the queues, time-out values when waiting for results, optimal number of threads to use, etc. This hasn't been given in the example code, but it's not difficult to add. What's more difficult is automatically creating the best values for a specific implementation.

Conclusion

Request threads allow a simple way to make asynchronous calls to shared resources. Delphi allows easy creation of such a method because of its strong support for COM, threading, and access to the Win32 API.

The source code, which is available for download (see end of article for details) includes a demonstration COM out-of-process server and a client test program. To enable threading on the server side, and so allowing the server to process multiple client calls simultaneously, the Borland-supplied source code thrddcf.pas has been employed. Without this the clients requests would be queued by the server and processed serially, one by one. This file may be found in the directory Demos\Midas\Pooler. Because of the multi-threaded nature of the server, it can't be used with Windows 95. It should be used on Windows NT only, whereas the client program can be used on Windows 95 or NT. Before using the client program, the server must be registered. This can be done by executing the server with the command line argument /regserver (or more easily from the Delphi IDE).

The files accompanying this article are available for download.

Michael J. Leaver is the Product Manager for the China Systems Prexim system. China Systems is a world leader in trade finance technology. Prexim is a workflow imaging solution for trade finance processing written using Delphi 4 C/S. Michael can be reached via e-mail at mailto:MJLeaver@csi.com.

Microsoft Internet Explorer

Top of page
 
Click HERE...FREE RoboHELP Starter Kit

Informant Communications Group

Informant Communications Group, Inc.
10519 E. Stockton Blvd., Suite 100
Elk Grove, CA 95624-9703
Phone: (916) 686-6610 • Fax: (916) 686-8497

Copyright © 1999 Informant Communications Group. All Rights Reserved. • Site Use Agreement • Send feedback to the Webmaster • Important information about privacy