Clever Foo

So I thought about this on my drive back from the gym today. What I believe Python needs for world domination is to drop its C runtime and adopt the mono runtime as the new “blessed by Guido” implementation. Why you may ask? It would bring together one of the best languages ever developed with one of the best runtimes in the world.

To put it differently, it gets the python guys to focus on the core language (which they are amazing at) and gets them away from being runtime developers (which they are not so good at) but, instead, Miguel and company are.

Think about it, Python users would get instant access to all of the CLI libraries (and vice versa), a fairly sane way to port C libraries to the CLR and instantaneous ass-kicking performance. For the mono guys, it would give it a jazzy new language to kick start the next generation of apps.
Don’t get me wrong, I’m not saying that there’s anything particularly bad about C# but, just like Java, in this day and age, it makes you feel like you are developing in 90’s technology.

Why forks don’t matter:

Before I get flooded with emails about IronPython, Jython, PyPy to IL compilation and all other nonsensical edgy stuff being done by people with too much free time, I’ll give you my “forks don’t matter” speech.

Let’s say If you and I were friends and I came to you and said…

Me:

You won’t believe it… I just bought last weekend’s Cards vs Eagles football on ebay, it’s freaking awesome!”

You:

Well, I have a football too, got it at WalMart and it is works just like yours

[Me punching you in the face]

But that’s the point, a functional copy of what I have, is not what I have. (IronP|J)ython is not Python, it’s something that behaves like Python but, fundamentally, it will never be the original. In practical terms, that’s how Linus can manage thousands of forks on the linux kernel and still hold the “one true” branch of the kernel – the forks don’t matter.

Does anyone know of such strange, yet delightful, gathering? I would like to see how things are going in mono land…

For instance in the code below:

using System.Threading;
public class foo {
	public static void Main() {
		new Timer(new TimerCallback(cb),null,200,5000);
		Thread.Sleep(Timeout.Infinite);
	}
	public static void cb(object state) {}
}

the current implementation of Threading.Timer will create a new thread for every new Timer object created. Why is this bad you might ask… well, primarily this is bad because we end up with a bunch of idle threads that do nothing but consume memory just to trigger the TimerCallback delegate at the end of its trigger period. Since each thread requires a chunk of stack space to exist, operating systems can only create a finite amount of threads before running out of memory… this makes threads somewhat of an expensive commodity to be just sitting idle for most of the time.

The patch that I have submitted changes this logic to utilize a single scheduler thread that is in charge of trigger all the timer events registered with its scheduler. The practical implications of this is that n timers can be created at the expense of a single thread.

Enough mambo-jambo, here’s an example of what I described above…

using System.Threading;
using System;

public class foo {
	public static void Main(string[] args) {

		int count = 1000;

		if (args.Length > 0) {
			count = Convert.ToInt32(args[0]);
		}

		Random rad = new Random();
		Console.WriteLine("Starting tests");
		Console.WriteLine("running test for {0} timers", count);

		DateTime start = DateTime.Now;
		for (int i=0;i < count;  i++) {
			new Timer(new TimerCallback(callback),null,rad.Next(5000),rad.Next(5000));
		}
		Console.WriteLine("{0} timers created in {1} msec",count,DateTime.Now.Millisecond - start.Millisecond);
		Thread.Sleep(Timeout.Infinite);

	}

	public static void callback(object state) {
	}
}

The code above just creates n timers (n being args[0]) as quickly as possible and under mono 1.1.13.4 the result is:

[terminal]
[rafael@stan tmp]$ mono timer2.exe 100
Starting tests
running test for 100 timers
100 timers created in 84 msec
[/terminal]

Now, under the single-thread scheduler Timer implementation the result is:

[terminal]
[rafael@stan tmp]$ /devel/bin/mono timer2.exe 100
Starting tests
running test for 100 timers
100 timers created in 66 msec
[/terminal]

So, slightly faster you might think right? Well, being faster is not the main goal of the Timer scheduler, it’s more like a side-effect under these conditions,, so let’s try a better test that really shows its capacity. For instance, changing our test to create a slightly grater number of timers, let’s say 10000 on mono 1.1.13.4:

[terminal]
[rafael@stan tmp]$ mono timer2.exe 10000
Starting tests
running test for 10000 timers
[/terminal]

causes the timer creation loop to never end because my x86 desktop (and pretty much all other systems out there) cannot really create 10k threads. Now, standing on the other side of the ring is our new Timer implementation which…

[terminal]
[rafael@stan tmp]$ /devel/bin/mono timer2.exe 10000
Starting tests
running test for 10000 timers
10000 timers created in 276 msec
[/terminal]

Can handle 10k timers without any issues. However, be warn! CPU consumption does become significant under 10k timers taking anywhere from 8 – 50% on my 1.8Ghz AMD cpu.

Well, I hope this helped you understand how mono’s Threading.Timer class works and how our single thread scheduler hopes to make it even better.

I just submitted a patch to upgrade monodoc to gtk-sharp-2.0 and thus finally put an end to all of the Pango related crashes in our beloved documentation browser… hopefully that will mean no more angry emails complaining that monodoc crashes too much. We will see.

Update.. here’s the proof:

So, I’m writing a little toy app (a single thread web server in c#) and I’m using the awesome “heap-buddy” tool to track heap usage… and this is what I see:
[terminal]
[rafael@salamandra Ophion.ZServer]$ heap-buddy /tmp/zserver.pf types fullname

Type # Total AvSz AvAge BT#
string 6462 563k 89.2 0.7 663
int 14731 172k 12.0 0.0 235
object[] 1043 68k 66.8 0.4 114
byte[] 1204 62k 53.5 0.1 98
char[] 107 51k 490.9 1.7 89
System.Text.StringBuilder 1294 30k 24.0 0.1 160

[/terminal]

about 90% of the 563k of string allocation came from log4net…. that is just plain awefull for a webserver in which every ms of waisted servicing time counts. I’m going to pursue this further an post a better analysis of the problem later, but for right now, all I can say is LAME!.

I commited the new bookmark code to monodoc yesterday. Even tho things are still pretty unpolished, I’m pretty happy with the outcome (give it a try and let me know!). Now I’m going to focus on tracking down some bugs that are making the browser crash while loading some “anchored” HTML. In the meantime, any feedback is much appreciated.

I’m submitting to the monodoc ([url]http://mono-project.com/Monodoc[/url]) mailing list a patch that gives persistent hierarchical bookmarks to monodoc. Hopefully this will get approved and merged upstream soon…
The patch took a lot longer than I first had planned since dealing with something like hierarchical bookmarks is not as trivial as one might think…
Since some poeple say a picture is worth 1000 words, so here it is:

I apologize for the ugly cropping but for some reason I could not get Gnome snapshot to work correctly.

- raf

Here’s a very simple wasy to generate a sha1 hash. You can run gensha1.exe like this:

[rafael@salamandra tmp]$ mono gensha1.exe rafael
generating sha1 for:rafael
2D-8D-59-6A-0B-97-56-9F-92-26-A8-C3-3E-D9-C6-DB-C8-D8-81-20
[rafael@salamandra tmp]$


and the source code looks like this:

using System;
using System.Security.Cryptography;

public class GenSha1 {
        public static string GenHash(string str) {
                byte[] bt = System.Text.Encoding.ASCII.GetBytes(str);
                SHA1 sha1 = new SHA1CryptoServiceProvider();
                return( BitConverter.ToString(sha1.ComputeHash(bt)) );
        }
        public static void Main(string [] args ) {
                string verb = args[0];
                Console.WriteLine("generating sha1 for:{0}",verb);
                Console.WriteLine(GenHash(verb));
                Environment.Exit(0);
        }
}