Concurrency Issues in Client/Server Applications

1. Concurrency Issues in Client/Server Applications Chapters 15,16, 28

2. Server Concurrency Control • Must choose between Iterative or Concurrent • How many clients may query server? • How often will clients Query? • What mode of query will be used? • How long will service take?

3. Server Concurrency Control • Must choose between Iterative or Concurrent • How many clients may query server? • How often will clients Query? • What mode of query will be used? • How long will service take? • If concurrent, must choose the level of concurrency • Apparent or real concurrency • Variable number of client threads • Fixed Maximum number of threads

4. Server Concurrency Models • Thread on demand • One thread per session • One thread per request

5. Server Concurrency Models • Thread on demand • One thread per session • One thread per request • Thread Pool • Create thread pool • Allocate threads to queries as needed. • When query has been served, return thread to pool • When all threads busy, queue or discard other queries

6. Socket Pre-allocation Techniques • Associate a thread (or a thread pool) with each different service • For TCP, associate each thread with the service socket, such that when a query arrives, one (and only one) thread will have accept unblocked, will create a new socket for this query, handle the query, and then return to accept. • For UDP, each request is a single datagram, which gets assigned to a single slave thread.

7. Concurrent Connection-oriented Master slave slave slave Port

8. Concurrent Connectionless Master slave slave slave Port

9. Delayed Concurrency in Servers • Concurrency helpful when the cost of a new process (thread) is less than service time for the request. • May vary by request. (Some database queries are short and others are long).

10. Delayed Concurrency in Servers • Concurrency helpful when the cost of a new process (thread) is less than service time for the request. • May vary by request. (Some database queries are short and others are long). • Start service iteratively and set timer. If timer expires, create new thread to finish service. • Start new threads based on service requested (where multiple services are available).

11. Concurrency in Clients • Why? • Easier to program, because concurrency forces modularity • Easier to maintain, also because of modularity • Can interact with several servers at once • Allows the user to interact with server, even when a call is blocked.

12. Concurrency in ClientsExample • Database query - “residents on Elm Street” • Response may be short (in Lone Jack).... or very long (in KCMO)

13. Concurrency in ClientsExample • Database query - “residents on Elm Street” • Response may be short (in Lone Jack).... or very long (in KCMO) • How does the user know when to abort a “failed” query? • Might query server for status... • Might terminate current query (if server is too slow) and restart on another server...

14. Functional Concurrency in Clients • Use multiple threads to perform different actions • One thread manages input (keyboard) • One thread manages input to socket • One thread manages output from socket • One thread manages control interaction with server • etc.

15. Concurrency in ClientsMultiple Servers • Allows multiple accesses at the same time...so that performance measures (round trip delay, lost packets, etc.) are all taken against the same network environment. • Faster than iterative model

16. Concurrency in ClientsIssues • Concurrency Model: May use real or apparent concurrency (single threaded or multi-threaded). • OS Support for concurrency: Some systems allow sharing memory between threads, while others may not. • Functional Decomposition: What operations might be done in parallel, and which need to be done sequentially?

17. Concurrency in ClientsSingly threaded • Allows memory sharing between connections • May (or may not) overload, depending on rate of data exchange with multiple connections • May deadlock!!

18. Concurrent Client (TCPtecho) #include <stdio.h>, <string.h>, <time.h>, <winsock.h> #define BUFSIZE 4096 /* write buffer size*/ #define CCOUNT 64*1024 /* default character count*/ #define WSVERS MAKEWORD(2, 0) #define MIN(x, y) ((x)>(y) ? (y) : (x)) #define USAGE "usage: TCPtecho [ -c count ] host1 host2...\n” struct hdat { char *hd_name; /* host name */ SOCKET hd_sock; /* host socket descriptor*/ unsigned hd_rc; /* recv character count */ unsigned hd_wc; /* send character count */ } hdat[FD_SETSIZE]; /* fd to host name mapping*/ char buf[BUFSIZE]; /* read/write data buffer */ void TCPtecho(fd_set *, int); int reader(struct hdat *, fd_set *); void writer(struct hdat *, fd_set *); long mstime(u_long *);

19. Concurrent Client (TCPtecho) void main(int argc, char *argv[]) { int ccount = CCOUNT, i, hcount = 0, fd; unsigned long one = 1; fd_set afds; WSADATA wsdata; if (WSAStartup(WSVERS, &wsdata)) errexit("WSAStartup failed\n"); FD_ZERO(&afds); for (i=1; i<argc; ++i) { if (strcmp(argv[i], "-c") == 0) { if (++i < argc && (ccount = atoi(argv[i]))) continue; errexit(USAGE); } /* else, a host */

20. Concurrent Client (TCPtecho) fd = connectTCP(argv[i], "echo"); if (ioctlsocket(fd, FIONBIO, &one)) { fprintf(stderr,"can't mark nonblocking (host %s): %d\n", argv[i], GetLastError()); continue; } hdat[hcount].hd_name = argv[i]; hdat[hcount].hd_sock = fd; hdat[hcount].hd_rc = hdat[hcount].hd_wc = ccount; ++hcount; FD_SET(fd, &afds); } TCPtecho(&afds, hcount); WSACleanup(); exit(0);}

21. Concurrent Client (TCPtecho) void TCPtecho(fd_set *pafds, int hcount) { fd_set rfds, wfds; /* read/write fd sets */ fd_set rcfds, wcfds; /* read/write fd sets (copy)*/ int fd, hndx, i; for (i=0; i<BUFSIZE; ++i) /* echo data */ buf[i] = 'D'; memcpy(&rcfds, pafds, sizeof(rcfds)); memcpy(&wcfds, pafds, sizeof(wcfds)); (void) mstime((u_long *)0); /* set the epoch */

22. Concurrent Client (TCPtecho) while (hcount) { memcpy(&rfds, &rcfds, sizeof(rfds)); memcpy(&wfds, &wcfds, sizeof(wfds)); if (select(FD_SETSIZE, &rfds, &wfds, (fd_set *)0, (struct timeval *)0) == SOCKET_ERROR) errexit("select failed: error %d\n",GetLastError()); for (hndx=0; hndx<hcount; ++hndx) { fd = hdat[hndx].hd_sock; if (FD_ISSET(fd, &rfds)) if (reader(&hdat[hndx], &rcfds) == 0){/* host done*/ for (i=hndx+1; i<hcount; ++i) hdat[i-1]=hdat[i]; hcount--; continue; } if (FD_ISSET(fd, &wfds)) writer(&hdat[hndx], &wcfds); } }}

23. Concurrent Client (TCPtecho) int reader(struct hdat *phd, fd_set *pfdset) { ... } void writer(struct hdat *phd, fd_set *pfdset) { ... } long mstime(u_long *pms) { ... }

24. Deadlock in Client/Server Systems • A situation where computation cannot proceed because a set or 2 or more components in the system is blocked and each component is waiting on another component in the set. • Cannot be released through an external input. • Often difficult to detect or prove, because it may depend on a particular, unusual sequence of events.

25. Deadlock Between Client & Server • May use request / response model to avoid deadlock. • Must ensure that protocol design is “tight”. Remove ambiguities in the design. • Assign responsibility for communication synchronization. (One side always starts)

26. Starvation in Client/Servers • Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources. • Results in unfair allocation of services

27. Starvation in Client/Servers • Occurs when some clients cannot obtain service because other clients or processes are monopolizing the server resources. • Results in unfair allocation of services • Delays waiting for a client to transmit (blocking on receive) can be managed through timeouts. • For busy / malicious clients, servers multithreading or non-blocking can be used.

28. Livelock in Clients / Servers • Like deadlock, livelock results from circular dependency among processes and resources. • System is locked into an endless self-generating sequence of commands that ultimately consumes all system resources.

29. Livelock in Clients / Servers • Like deadlock, livelock results from circular dependency among processes and resources. • System is locked into an endless self-generating sequence of commands that ultimately consumes all system resources. • Example: • File Server with time stamp logging. • Livelock occurs if a remote call to time is used • First file request comes in. • Time stamp requires time (which generates an incoming message to file server). • Incoming time response looks like a file request, which requires a new time stamp.....

30. Planning for Concurrency • Consider what system resources should be consumed by the server • Consider what the demand for the service(s) is • Provide limits (threads, processes, memory) that are appropriate. • Incorporate logging into the server to track program problems, congestion, etc. • Consider “livelock” problems when working with multiple services.