Fazal Majid's low-intensity blog

The current version of Internet Protocol (IP), the communications protocol underlying the Internet, is version 4. In IPv4, the address of any machine on the Internet, whether a client or a server, is encoded in 4 bytes. Due to various overheads, the total number of addresses available for use is much less than the theoretical 4 billion possible. This is leading to a worldwide crunch in the availability of addresses, and rationing is in effect, specially in Asia, which came late to the Internet party and has a short allocation (Stanford University has more IPv4 addresses allocated to it than the whole of China).

Internet Protocol version 6, IPv6, quadrupled the size of the address field to 16 bytes, i.e. unlimited for all practical purposes, and made various other improvements. Unfortunately, its authors severely underestimated the complexity of migrating from IPv4 to IPv6, which is why it hasn’t caught on as quickly as it should have, even though the new protocol is almost a decade old now. Asian countries are leading in IPv6 adoption, simply because they don’t have the choice. Many people make do today with Network Address Translation (NAT), where a box (like a DSL router) allows several machines to share a single global IP address, but this is not an ideal solution, and one that only postpones the inevitable (but not imminent) reckoning.

One misconception, however, is that that the slow pace of the migration is somehow related to the fact you get your IP addresses from your ISP, and don’t “own” them or have the option to port them the way you now can with your fixed or mobile phone numbers. While IPv6 greatly increases the number of addresses available for assignment, this will not change the way addresses are allocated, for reasons unrelated to the address space crunch.

First of all, nothing precludes anyone from requesting an IPv4 address directly from the registry in charge of their continent:

• ARIN in North America and Africa south of the Equator
• LACNIC for Latin America and the Caribbean
• RIPE (my former neighbors in Amsterdam) for Europe, Africa north of the Equator, and Central Asia
• APNIC for the rest of Asia and the Pacific.

That said, these registries take the IP address shortage seriously and will require justification to grant the request. Apart from ISPs, the other main kind of allocation recipients are large organizations that require significant numbers of IP addresses (e.g. for a corporate Intranet) and that will use multiple ISPs for their Internet connectivity.

The reason why IP addresses are allocated mostly through ISPs is the stability of the routing protocols used by ISPs to provide global IP connectivity. The Internet is a federation of independent networks that agree to exchange traffic, sometimes for free (peering) or for a fee (transit). Each of these networks is called an “Autonomous System” (AS) and has an AS number (ASN) assigned to it. ASNs are coded in 16 bits, so there are only 65536 available to begin with.

When your IP packets go from your machine to their destination, they will first go through your ISP’s routers to your ISP’s border gateway that connects to other transit or final destination ISPs leading to your destination. There usually are an order of magnitude or two fewer border routers than interior routers. The interior routers do not need much intelligence, all they need to know is how to get their packets to the border. The border routers, on the other hand, need to have a map of the entire Internet. For each block of possible destination IP addresses, they need to know which next-hop ISP to forward the packet on to. Border routers exchange routing information using the Border Gateway Protocol, version 4 (BGP4).

BGP4 is in many ways black magic. Any mistake in BGP configuration can break connectivity or otherwise impair the stability of vast swathes of the Internet. Very few vendors know how to make reliable and stable implementations of BGP4 (Cisco and Juniper are the only two really trusted to get it right), and very few network engineers have real-world experience with BGP4, learned mostly through apprenticeship. BGP4 in the real scary world of the Internet is very different from the safe and stable confines of a Cisco certification lab. The BGP administrators worldwide are a very tightly knit cadre of professionals, who gather in organizations like NANOG and shepherd the Net.

The state of the art in exterior routing protocols like BGP4 has not markedly improved in recent years, and the current state of the art in core router technology just barely keeps up with the fluctuations in BGP. One of the control factors is the total size of BGP routing tables, which has been steadily increasing as the Internet expands (but no longer exponentially, as was the case in the early days). The bigger the routing tables, the more memory has to be added to each and every border router in the planet, and the slower route lookups will be. For this reason, network engineers are rightly paranoid about keeping routing tables small. Their main weapon consists of aggregating blocks of IP addresses that should be forwarded the same way, so they take up only one slot.

Now assume every Internet user on the planet has his own IP address that is completely portable. The size of the routing tables would explode from 200,000 or so today to hundreds of millions. Every time someone logged on to a dialup connection, every core router on the planet would have to be informed, and they would simply collapse under the sheer volume of routing information overhead, and not have the time to forward actual data packets.

This is the reason why IP addresses will continue to be assigned by your ISP: doing it this way allows your ISP to aggregate all its IP addresses in a single block, and send a single route to all its partners. Upstream transit ISPs do even more aggregation, and keep the routing tables to a manageable size. The discipline introduced by the regional registries and ISPs is precisely what changed the exponential trend in routing table growth (one which even Moore’s law would not be able to keep up with) to a linear one.

It’s not as if this requirement is anti-competitive, unlike telcos dragging their feet on number portability – the DNS was precisely created so users would not have to deal with IP addresses, and can easily be changed to point to new addresses in the event of a change of IP addresses.

After a comments thread on a request in Wilfredo Sanchez’ blog, including valuable contributions by Bob Ippolito, I have updated my threadframe module to leverage new thread tracking functionality added in Python 2.3. The module now allows Python 2.3 users to extract the stack frame of any given thread using its thread ID.

Note: threadframe is obsolete. Python 2.5 and later include a function sys._current_frames() that does the same thing. Threadframe is only useful for Python 2.2 through 2.4.

Rationale

I was encountering deadlocks in a multi-threaded CORBA server (implemented using omniORB). Debugging using GDB gave me too low-level information, and what I needed was an equivalent of the GDB command “info threads”. There was no such facility available from within Python’s standard library, so I rolled my own.

David Beazley added advanced debugging functions to the Python interpreter, and they have been folded into the 2.2 release.

I used these hooks to build a debugging module that is useful when you are looking for deadlocks in a multithreaded application. It basically has a single function that will return a list of the stack frames for all Python interpreter threads in the process.

Guido van Rossum added in Python 2.3 the thread ID to the interpreter state structure, and this allows us to produce a dictionary mapping thread IDs to frames.

This functionality is now integrated in Python 2.5’s batteries-included sys._current_frames() function.

Of course, I disclaim any liability if this code should crash your system, erase your homework, eat your dog (who also ate your homework) or otherwise have any undesirable effect.

Building and installing

Python 2.2 or later is required. Thread ID to frame dictionary extraction is only available in Python 2.3 and later, and will generate a NotImplementedError if used from 2.2.

Download the source tarball threadframe-0.2.tar.gz. You can use the Makefile or directly with the setup.py script. I have built and tested this only on Solaris 8/x86 and Windows 2000, but the code should be pretty portable. There is a small test program test.py that illustrates how to use this module to dump stack frames of all the Python interpreter threads. A sample run is available for your perusal.

For Windows users, I have available pre-compiled binaries, built using Mingw32 and GCC 2.95.2. Just copy the file threadframe.pyd in any location in your Python path and you should be able to run the test script test.py.

License

This code is licensed under the same terms as Python itself.

Change history

Release 0.2 (2004-06-10)

Distutils based setup.py contributed by Bob Ippolito. Bob also noticed that thread_id was added to the Python interpreter state, and contributed a patch to get a dictionary mapping thread_ids to frames instead of a list.

Release 0.1 (2002-10-11)

Initial release for Python 2.2: threadframe-0.1.tar.gz

Apple introduced the Airport Express today, surprising observers who expected product announcements to be on hold until the WWDC conference in San Francisco later this month. Apple-watching is a surprise-fraught art not unlike Kremlinology used to be, with the added risk of cease-and-desist letters by the notoriously secretive and litigious company.

The Airport Express is a compact little wireless network in a box, offering an IEEE 802.11g WiFi access point cum router, an Ethernet port, an audio port to stream audio (interestingly, it supports both conventional electrical line-level output as well as Toslink optical in the same jack), and a USB port to allow printer sharing (no word on whether this also allows scanner sharing the way Keyspan’s USB server does).

This unit replaces 2 or 3 boxes (and their associated wall warts), is relatively inexpensive at $129, and will no doubt become as popular and widely (yet poorly) imitated as the iPod was in its day, specially given it can be used by Windows PCs. If I did not already have a Slimdevices Squeezebox (with beta support for the Apple lossless encoder), I might have be tempted, in spite of the lack of a display or remote control.

I am not all that fond of the wall-wart concept, but the plug can be removed and replaced with a standard IEC-320-C7 cable (which can certainly be found far cheaper than the ridiculously expensive $39 Apple charges for them), or even powered from Ethernet using the new power-over-Ethernet standard 802.3af (the USB port is disabled in that case), a nice touch that exemplifies Apple’s attention to detail. As a side note for those of you who have a hard time coping with wall warts, I highly recommend the Power Strip Liberator Plus, a simple but highly effective solution to the problem of clogged power strips.

That said, there is one port missing, one that would have turned the Airport Express from a well-designed piece of electronics into a visionary product: a phone jack. A RJ-11 jack that can be plugged into a phone line (FXO) or into which a phone can be plugged (FXS) would bridge one of the few remaining domains not covered by Apple’s digital hub (the other one being TV). With iTunes AV, Apple has a very capable Voice over IP (VoIP) client, but no way to interface it to legacy POTS (Plain Old Telephone System) networks. I am not sure if this is deliberate and if they want to introduce this as a value-added feature to their .Mac Internet services suite, but Apple has lacked decent telephony product since the introduction of the Geoport ten years ago.

It should be straightforward to add telephony software to a Mac and have it able to act as an intelligent voice-mail or IVR system (forwarding voice mails via email the way Panther’s Fax feature can with faxes). Computer-Telephony Integration, widespread in the PC world, is also an essential feature for many enterprise applications (think call centers or CRM). Many small businesses use Macs because they cannot afford full-time IT staff to baby-sit Windows machines. Offering them an integrated telephony solution would be a very attractive proposition.

Ghirardelli is the best-known chocolate maker from San Francisco, but by no means the only one. The Bay Area is very serious about food, and boasts many fine chocolatiers such as Guittard, Scharffen-Berger, Joseph Schmidt, and Michael Recchiuti, all of which uphold a much higher standard of quality than Ghirardelli (while not inedible dreck like Hershey’s, Ghirardelli is over-sweet and fairly lackluster).

Guittard is not as well known, as they used not to sell retail (their chocolate is used, among others, by See’s Candies and Boudin Bakery, and I once had a wonderful cherry and Guittard chocolate cake at Eno in Atlanta). This changed when they recently introduced a line of premium chocolates, named after the firms’s French founder, Etienne Guittard.

They probably don’t have an extensive distribution network yet, but their products are starting to trickle into finer San Francisco groceries like my neighborhood one, Lebeau Nob Hill Market (“People in the Know / Shop at Lebeau”).

I bought a 500g box of their “Soleil d’Or” milk chocolate, packaged as a box of “wafers” (little quarter-sized pieces reminiscent of Droste Pastilles). In this form, it is intended for cooking, but the bite-sized wafers are also perfect for snacking. It has a relatively high cocoa content for milk chocolate (38%, the usual is more like 32%), which gives it a satisfying taste that lingers in the mouth. This chocolate is also well balanced, it does not have the malty harshness of Scharffen-Berger milk chocolate or the milky aftertaste of Valrhona “Le Lacté”. In fact, it comes close to my personal favorite, Michel Cluizel “Grand Lait Java”, no small achievement, specially when you consider the difference in cocoa content (38% vs. 50%) and the price difference ($9 for a 500g box vs. $5 for a 100g tablet).

Update (2004-12-30):

Guittard updated their packaging (shown right). The newer one is more classy and eschews the pretentious “Soleil d’Or” and “Collection Etienne” labels, but the chocolate itself is unchanged. The box is also slightly lighter (1lb or 454g vs. 500g for the older one, i.e. a 10% price increase…), but at $9.99/lb, you are still paying Lindt prices for near Cluizel quality