IT

I seem to be late to this party, but one of the security updates for Windows XP (.NET 3.5) silently installs a Firefox plugin that:

1. tells every web server you visit which version of the .NET framework you have, in my case

<tt>Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)</tt>

2. allows websites to install software on your desktop using ClickOnce, a mechanism so abysmally stupid in its insecurity it gives ActiveX a run for its money.

The reason why Microsoft is doing this is to increase penetration of its also-ran Silverlight competitor to Flash for the 20-30% of Windows users who use Firefox instead of Internet Exploder. To make matters worse, the plugin uninstallation button is grayed out. A Microsoft staffer has published instructions on removing this on his blog.

This behavior is of course completely unacceptable. Perhaps Adobe will now join the line of Microsoft-bashers at the European Commission.

Update (2009-10-18):

Good news: Mozilla responded quickly to block this piece of malware. That should also disable Silverlight altogether. Two birds with one stone.

I decided to take action and wrote a letter (PDF) to EC Commissioner Neelie Kroes, apparently the only person in the world who has the cojones to confront Microsoft about its practices.

I attended the OpenSolaris Storage Summit in San Francisco. Unsurprisingly, SSDs dominated the proceedings. Sun is touting its hybrid storage pool approach, where SSDs are used to accelerate the ZFS Intent Log (ZIL), a journal, and the L2ARC read cache, to get most of the benefits of SSDs without having to store an entire dataset in expensive SSDs. The presentations on the page are worth having a look into.

Andrew Leventhal commented on how using disk-oriented APIs to access SSDs makes as little sense as using cassette tape adapters to connect an iPod to a car stereo. Sure, it works, sort of, as a short-term compatibility mode, but you are not using the potential of the device. Fusion IO makes SSDs that are connected to the PCIe bus directly and get 5x better performance than the best SATA drive, the Intel X25-E, but it uses proprietary drivers. In one of the video presentations distributed in the conference, Andy Bechtolsheim mentioned he thought a memory channel model is a better fit for the hardware, as the block-device model and all the legacy assumptions behind it is holding back databases from using the full power of SSDs.

A modest suggestion – there is such an API available already: memory-mapped files. In an ironic full circle, an abstraction meant to simulate random access memory on rotating media is actually the best fit to the actual device characteristics. There are also obvious benefits such as DMA.

A few months ago, my former EuroNet colleague Niels Bakker was visiting SF. He works for AMS-IX, the Amsterdam Internet Exchange, which is the world’s largest by volume of traffic, and mentioned they work with a mere 25 employees.

Today, I was attending a presentation by Don MacAskill, the CEO of photo-sharing service SmugMug. He has 2 sysadmins managing well over 300 servers.

At the same time, you hear about the astronomical costs of corporate IT departments: data centers that routinely cost hundreds of millions of dollars, plethora of staff delivering pitiful results and systems that have user interface even a novice coder could beat in a single day of coding..

Why is this so?

You have the usual suspects:

• Dysfunctional top-down corporate cultures, specially when decisions are made on political grounds, i.e. which vendor plays golf with the CIO (or CEO). Often the grunts on the ground know what needs to be done, but are defeated and dispirited by years of failing to budge the bureaucracy.
• Use of poorly manageable software like Windows
• A culture of fire-fighting that eschews automation.
• Risk aversion leading to excessive redundancy. I still cannot understand how Red Hat gets away with its outrageous pricing on RHEL 7.
• In most large corporations the concentration of financial responsibility in a separate department means most employees, including sysadmins, do not feel empowered or responsible for looking out for the company’s money. The bean counters, on the other hand, lack the knowledge required to find the cost savings.

One would think the new economic reality would force a reckoning. It would stand to reason that most companies would institute policies of procuring open-source software first, and only purchase commercial software on an case-by-case exception basis, with tough questions asked. This is still novel enough to make the news.

Single-thread processor performance has stalled for a few years now. Intel and AMD have tried to compensate by multiplying cores, but the software world has not risen to the challenge, mostly because the problem is a genuinely hard one.

Shell scripts are still usually serial, and increasingly at odds with the multi-core future of computing. Let’s take a simple task as an example, converting a large collection of images from TIFF to JPEG format using a tool like ImageMagick. One approach would be to spawn a convert process per input file as follows:

#!/bin/sh
for file in *.tif; do
  convert $file `echo $file|sed -e 's/.tif$/.jpg/g' &
done

This does not work. If you have many TIFF files to convert (what would be the point of parallelizing if that were not the case?), you will fork off too many processes, which will contend for CPU and disk I/O bandwidth, causing massive congestion and degrading performance. What you want is to have only as many concurrent processes as there are cores in your system (possibly adding a few more because a tool like convert is not 100% efficient at using CPU power). This way you can tap into the full power of your system without overloading it.

The GNU xargs utility gives you that power using its -P flag. xargs is a UNIX utility that was designed to work around limits on the maximum size of a command line (usually 256 or 512 bytes). Instead of supplying arguments over the command-line, you supply them as the standard input of xargs, which then breaks them into manageable chunks and passes them to the utility you specify.

The -P flag to GNU xargsspecifies how many concurrent processes can be running. Some other variants of xargs like OS X’s non-GNU (presumably BSD) xargs also support -P but not Solaris’. xargs is very easy to script and can provide a significant boost to batch performance. The previous script can be rewritten to use 4 parallel processes:

#!/bin/sh
CPUS=4
ls *.tif|sed -e 's/.tif$//g'|gxargs -P $CPUS -n 1 -I x convert x.tif x.jpg

On my Sun Ultra 40 M2 (2x 1.8GHz AMD Opterons, single-core), I benchmarked this procedure against 920MB of TIFF files. As could be expected, going from 1 to 2 concurrent processes improved throughput dramatically, going from 2 to 3 yielded marginal improvements (convert is pretty good at utilizing CPU to the max). Going from 3 to 4 actually degraded performance, presumably due to the kernel overhead of managing the contention.

Another utility that is parallelizable is GNU make using the -j flag. I parallelize as many of my build procedures as possible, but for many open-source packages, the usual configure step is not parallelized (because configure does not really understand the concept of dependencies). Unfortunately there are too many projects whose makefiles are missing dependencies, causing parallelized makes to fail. In this day and age of Moore’s law running out of steam as far as single-task performance is concerned, harnessing parallelism using gxargs -P or gmake -j is no longer a luxury but should be considered a necessity.

My primary computer is a dual 2GHz PowerMac G5 until I can upgrade it with a Nehalem Mac Pro, most likely around the end of the year or early next year. I bought it in 2004, along with a 23″ Apple Cinema HD (the old pinstripe plastic bezel kind with an ADC connector). Unfortunately, about a year ago the CCFL backlight on the monitor started turning pink from old age, and thus unusable in a properly color-managed photographic workflow.

I used that as an excuse to splurge on a humongous (and agoraphobia-inducing) HP LP3065 30 inch LCD monitor after reading the glowing reviews. The two features that sold me were the enhanced color gamut (the only way to improve that would be to get a $6000 Samsung XL30, something I am not quite prepared to do), and the fact it has 3 built-in DVI ports, so it can easily be shared by multiple computers (assuming they support dual-link DVI, which unfortunately my basic spec Sun Ultra 40 M2 does not). The fact it was 25% less expensive than the Apple 30″ Cinema Display helped, of course.

About 6 months ago, I discovered there was a fine pink vertical line running across the entire height of the monitor, roughly 25 centimeters from the left. Since I primarily use that monitor for photo (the primary monitor for Mail, web browsing or terminals remains the Apple), at first I worried there was a defect with my camera. I managed to reproduce the problem with my MacBook Pro (they have dual-link DVI, unlike lesser laptops), and called HP support (the 3 year HP warranty was also an important consideration when I purchased).

My first support call in November 2007 went well, and the tech told me I would be contacted to arrange for an on-site exchange. This is a seriously heavy monitor and I did not relish the idea of lugging it back to FedEx, so getting premium support for a business-class monitor sounded an attractive proposition. Unfortunately, they never did call back, and as I had other pressing matters to attend to involving international travel, I just put it out of my mind (it is a very subtle flaw that is not even always visible).

I only got around to calling them back a few weeks ago. Unlike in November, I was given the run-around with various customer service reps in India until I was finally routed to a pleasant (and competent) tech in a suburb of Vancouver (the US dollar going in the direction it is, you have to wonder how much longer before HP outsources those call centers back to the US). The problem is not with Indian call centers, in any case, all but one of the CSRs were very polite (I suspect Indians learn more patience as they grow up than pampered Americans or Europeans would). The problem is poorly organized support processes and asinine scripts they are required to go through if they want to keep their jobs. In any case, the Canadian rep managed to find the FRU number and also told me someone would call to schedule an appointment. Someone did call this time, to let me know the part was back-ordered and they would call me when it becomes available.

This morning, as I was heading for the shower, my intercom buzzed. It was a DHL delivery man with the replacement monitor. I had to open the door to him in my bath robe. Naturally, nobody at HP bothered to notify me and had I left earlier, I would have missed him altogether.

One of the great things about Apple products is that if you live near an Apple store, you can just stop by their pretentiously-named Genius bars and get support for free (though not free repairs for out-of-warranty products, obviously). I now have a fully working HP monitor again, so I suppose I can’t complain too loudly, but the Apple monitor with the sterling support looks like the true bargain in hindsight.