December 2007 - Posts
Yeah, I know now there are some non-technical readers subscribed to my blog 
. So I just thought I would share something useful for them as well. Although heading down under was just great, leaving our parents, friends and relatives was not that easy for all of us. Therefore since very first day, I'm trying my best to keep up the close ties between us as much as possible. Here are some of the tools/services I find really useful.
Skype: Allows you to make voice calls to PCs (free) and phones. So far I've found that calling an SLT phone is a lot better than calling any other operator.
WOW SMS: Provides a free service to SMS some of Sri Lankan mobile carriers. Although there is a daily limit (15 messages per day) this is a fantastic way to exchange short messages. When you register, the service also provides an inbox number for you and your contacts can reply to that number. You just have to logon to the service to read incoming messages.
Kapruka: Great way to purchase and deliver gift items for them (Well... if you want to send something for us, you are welcome to order it from http://www.ebay.com.au/ for now 
. I will publish more links later).
BTW: I'm still trying to find a Sinhala webmail solution (AFAIK, gmail, yahoo and hotmail do not support this). If you know any, please drop me a note.
I invested a lot of time, money and gave a very careful thought when I wanted to move to Australia. After all it seemed to be the best option in mid and long term basis. The decision certainly affected my personal, professional and career life and I'm certainly enjoying the consequences now.
Over the past few years I was really missing a good broadband connection. Although I was fortunate to get access to MSDN library, Channel 9, .NET rocks, Codeplex and various other technical resources, being active in those spaces was a real bummer with my poor connection. But today I downloaded Windows Server 2008 RC1 (yeah with HyperV ) just in about 90 minutes (every time I tried something similar earlier it took nearly 40 hours or more). I don't think that I would wait for my MSDN DVDs to arrive any more.
So that's just another thing to be a happy man this Christmas.
A reader of my IOCP performance post recently sent me some interesting questions. I actually found them really, really interesting and thought I'd share my answers here instead of just replying to his email.
Q: I understand that at any given point there will be N number of threads associated with an IO Completion Port. Where N is number of Proccessors. If I call GetAvailbaleThreads it returns 25 for Worker Threads and 1000 IO threads. My question is why do we have such a high value for IO Threads as compared to worker threads, when Completion Port can only perform tasks in a sequential order in a queue.
A: You should not get confused with the numbers returned by GetAvailableThreads and the actual number of active IO worker threads. Numbers 25 and 1000 are merely tweaks made in the runtime. These numbers just represent the max number of non IO and IO worker threads that thread pool can spin up concurrently. The theory behind these numbers goes like this. Normal thread pool worker threads are not capable of preserving one thread per core per execution quantum invariant. Therefore if you queue 10 presumably long running items to the thread pool, all ten items will spin a new thread will be active in the thread pool. This also means if you queue CPU intensive work items, you will end-up with poor performance due to high amount of context switching etc. That's why runtime try to mitigate this by setting max number of threads in the thread pool to 25 by default. On the other hand, if you have 25 long running IO operations, you are out of luck unless you change the defaults. Your CPU will still be idle while bunch of threads in the thread pool are waiting for IO.
IO worker threads on the other hand make sure that there are only N number of threads active at any given time (thus preserve the one thread per core per execution quantum invariant). Where N is the number of processors/cores (It might be slightly above N in most circumstances as I explained in the original post, but that's absolutely negligible). So the thread pool can actually spin up as much as threads (1000 by default) if necessary to wait on the IOCP for the next work item. So let's say you queue 25 work items to IO worker threads to wait on some long running IO and then you queue another CPU intensive work. You still get your CPU intensive work done because all 25 IO worker threads are waiting for IO thus making the CPU idle.
Q: My understanding is that when a request is sent to Completion Port , it will finish that task first then move on to the next task. How would this scale if I make two request to download a 700mb files and third request to download 7mb file. Will the third request wait unless request 1 and 2 is finished? I hope not but my understanding is it will.
A: I think my answer to the first question explains this as well. Key thing here is, if you do some blocking work in an IO worker thread, Windows scheduler notifies the IOCP that one active thread went to inactive state thereby making the next thread waiting on the IOCP to go and pick up the next available work item. So if we put this to the sample context explained in the question, I would say, you can make IO worker threads download all 700MB, 700MB and 7MB files and whichever finishes first will probably finish its work first.
Q: How do we use Overlapped Object. I see we can specify offsets, and event handler but where do we specify the IO Handle based on which the operation should be made?
A: All this time I was talking about using IOCP to efficiently use CPU as opposed to how you would do this with regular thread pool work threads. In other words it can be thought as an efficient scheduling method. However, if you want to do asynchronous IO, you can just use the async operations in the .NET IO API (i.e. BeginXxx and EndXxx methods). Almost all of them use IOCP behind the covers. However, if you are still fiddling your keyboard to get your C program up and running, you can use CreateIoCompletionPort API and pass your IO handle to its first parameter . Then you can use IO API (ReadFileEx for example) to pass the necessary async IO information using OVERLAPPED structure.
HANDLE CreateIoCompletionPort(
HANDLE FileHandle,
HANDLE ExistingCompletionPort,
ULONG_PTR CompletionKey,
DWORD NumberOfConcurrentThreads
);
Q: Is IOCP is just maintaining a queue and after it deques it calls the return function with offsets to perform the operation?
A: The underlying Windows API actually does not use a function pointer per se. It's implemented in .NET framework thread pool (AFAIK). The original API uses a key (ULONG_PTR) based mechanism to correlate the work items when then return from IOCP.
Unlike the server application developers who used Winsock API directly in their code; now most of us rely on different abstractions like WCF to deal with Winsock API (and yes, this is a lot better). Each connection opened between the client and service consumes their resources (each tcp connection requires space for its kernel level structures for example). Therefore it's nice to have a good understanding of when they come and go if you want to build scalable solutions. Consequently I thought I would share some of my experience in connection management with you and this is my first attempt to discuss some subtleties in http connection management in services and clients.
When you use http transport WCF uses System.Net API in client and http.sys API in the server internally for the underlying communication. i.e. WCF does not directly deal with sockets like it does in tcp transport.
Let's first take a look at what happens in the client side. When you invoke a service method in the proxy class instance, underlying http transport channel uses the API in System.Net.HttpWebRequest class to send the http request to the server. It also creates a System.Net.ServicePoint to manage the connections to the specific server resource. You can think of this ServicePoint instance as a connection pool for http connections. Therefore the actual lifetime of a connection depends on the corresponding ServicePoint settings. For example, the first impression you get on the following code snippet is that the underlying connection is closed and all resources are claimed after the last line.
string uri = "http://localhost:8000/service";
BasicHttpBinding clientBinding = new BasicHttpBinding();
ChannelFactory<IFoo> cf = new ChannelFactory<IFoo>(clientBinding, uri);
IFoo f = cf.CreateChannel();
f.DoSomething();
((IClientChannel)f).Close();
cf.Close();
But if you take a closer look using a tool like tcpview.exe you will notice that the connection will remain open for little bit longer even after closing both channel and channel factory.
Reason for this is the connection pooling done by the corresponding ServicePoint as I mentioned before. By default ServicePoint sets the MaxIdleTimeout of a connection to 100000 milliseconds and the connections will be closed after 10000 milliseconds without any activity. So in some cases you may want to tweak this value to reach the optimum results. For example, if you have a client that connects to the service once in a blue moon and makes only one or two calls to the service, you might want close the connection and release the resources as soon as you are done. You can do this by setting the desired timeout value to ServicePointManager.MaxServicePointIdleTime property. However it's important to be aware of two things before hand. 1. You have to set this value prior to creating any channels or channel factories in your code. 2. this value will be used as the idle timeout for all ServicePoints created afterwards (even for different endpoints). One way to eliminate these two would be getting your hands on the specific ServicePoint as shown below. Then again, you must ensure that you call FindServicePoint method after calling the service method as runtime creates this ServicePoint during the first call to the service.
string uri = "http://localhost:8000/service";
BasicHttpBinding clientBinding = new BasicHttpBinding();
ChannelFactory<IFoo> cf = new ChannelFactory<IFoo>(clientBinding, uri);
IFoo f = cf.CreateChannel();
f.DoSomething();
ServicePoint sp = ServicePointManager.FindServicePoint(new Uri(uri));
sp.MaxIdleTime = 1000 * 10; // Set idle timeout to 10 seconds
((IClientChannel)f).Close();
cf.Close();
Having this redundant code would be probably painful. So I could compile it to a little WCF extension hooked up as a custom operation behaviour like this.
[AttributeUsage(AttributeTargets.Interface | AttributeTargets.Method)]
public class SetHttpConnectionIdleTimeoutAttribute : Attribute, IOperationBehavior
{
private int timeout;
public int Timeout
{
get
{
return timeout;
}
set
{
if (value < 0)
{
throw new InvalidOperationException("Timeout must be a positive integer.");
}
timeout = value;
}
}
public void AddBindingParameters(OperationDescription operationDescription,
BindingParameterCollection bindingParameters)
{
// nop
}
public void ApplyClientBehavior(OperationDescription operationDescription,
ClientOperation clientOperation)
{
ConnectionManager cm = AddOrFindConnectionManager(clientOperation.Parent);
cm.EnlistOperation(clientOperation, timeout);
}
public void ApplyDispatchBehavior(OperationDescription operationDescription,
DispatchOperation dispatchOperation)
{
// nop
}
public void Validate(OperationDescription operationDescription)
{
// nop
}
private ConnectionManager AddOrFindConnectionManager(ClientRuntime clientRuntime)
{
ConnectionManager cm = null;
foreach (IClientMessageInspector mi in clientRuntime.MessageInspectors)
{
if (typeof(ConnectionManager) == mi.GetType())
{
cm = (ConnectionManager)mi;
break;
}
}
if (cm == null)
{
cm = new ConnectionManager();
clientRuntime.MessageInspectors.Add(cm);
}
return cm;
}
}
internal class ConnectionManager : IClientMessageInspector
{
class RequestInfo
{
private string action;
private Uri remoteAddress;
public RequestInfo(string action, Uri remoteAddress)
{
this.action = action;
this.remoteAddress = remoteAddress;
}
public string Action
{
get { return action; }
set { action = value; }
}
public Uri RemoteAddress
{
get { return remoteAddress; }
set { remoteAddress = value; }
}
}
Dictionary<string, int> registeredOperations = new Dictionary<string, int>();
public void AfterReceiveReply(ref Message reply, object correlationState)
{
RequestInfo ri = (RequestInfo)correlationState;
ServicePoint servicePoint = ServicePointManager.FindServicePoint(ri.RemoteAddress);
if (servicePoint != null)
{
Debug.Assert(registeredOperations.ContainsKey(ri.Action));
int timeout = registeredOperations[ri.Action];
servicePoint.MaxIdleTime = timeout;
}
}
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
return new RequestInfo(request.Headers.Action, channel.RemoteAddress.Uri);
}
public void EnlistOperation(ClientOperation clientOperation, int timeout)
{
if (!registeredOperations.ContainsKey(clientOperation.Action))
{
registeredOperations.Add(clientOperation.Action, timeout);
}
}
}Now this extension can be used as an attribute in your operation contracts as shown below.
[ServiceContract]
interface IFoo
{
[OperationContract]
[SetHttpConnectionIdleTimeout(Timeout=5000)]
void DoSomething();
}If you want to close your connection as soon as you are done, make sure that you set your timeout to 0. But once again, keep in mind that reducing the timeout without a careful thought can surely lead into poor performance. Do it only if you really want to do it.
Alright... we talked about how we can manage the connections on the client side. So what about the service side? In some cases you may want to reduce the idle timeout in the server. This is sometimes essential when you are not controlling clients (the problem that drew my attention to this post was exactly that ). In the service side, I still could not find a way to get the hands on the underlying connections like we did in the client side. However, you could always configure the HTTP runtime using provided tooling. For example, in Windows Vista you can use following netsh command to reduce the timeout to 20 seconds from its 120 seconds default value.
netsh http add timeout timeouttype=idleconnectiontimeout value=20
Furthermore you could use the following command to view the existing timeouts.
It's a bit unfortunate that changing idle timeout using netsh causes the whole http runtime to change though. I really wish that there will be a feature in the future to control this per registered URL basis.
Last time I p
osted about my last day at thinktecture and today I proudly announce that I joined Readify last week. As a result I and my family arrived in Canberra last week. I'm truly honoured to be a part of one of the greatest consulting teams in this part of the world. I met a few guys over past few days and they are just fantastic and I'm quite convinced that there is a whole lot of fun ahead.
In most cases when you switch between jobs it's highly unlikely that you get to do what you want to do. However, I'm pretty delighted that I will get to continue my professional career in the areas that I'm mostly interested in. So you can definitely expect more and more ramblings on my WCF, concurrency and web services adventures. Other than that I'm also hoping to post remarkable things come along the way to settling in for a new life down under.
Last but absolutely not least I would like to thank Readify for making everything so comfortable for me. I never felt like an alien since I arrived in Sydney airport.
BTW: For those who know me, if you are planning to visit down under, try to buzz me before you come. We can probably catch up for couple of drinks together.