If WordPress updates hang on a 64-bit OS

The WordPress instance running this site was no longer able to automatically update plugins (and presumably not the core either) after I upgraded from a 32-bit to a sparkling fresh 64-bit PHP install at Joyent. It would start the update, and show a spinning logo and then just hang.

After much debugging, I found out the problem is that the class-pclzip.php that is responsible for unzipping was failing silently with the message:

Downloading update from http://downloads.wordpress.org/plugin/yet-another-related-posts-plugin.3.5.2.zip

Unpacking the update…

Abort class-pclzip.php : Missing zlib extensions

This isn’t terribly helpful, but digging in, it turns out that class depends on the PHP zlib module, and on 64-bit operating systems (more precisely, operating systems with 64-bit large file support enabled), zlib.h #defines gzopen to be gzopen64. PHP does not protect itself adequately and thus the PHP function gzopen gets renamed gzopen64 as well, this throwing class-pclzip.php for a loop, along with a number of other systems like PEAR.

Fixing this requires recompiling PHP. Ubuntu Karmic includes a work-around, but I run Solaris and build from source, so I contributed a patch filed under bug #53829.

Automattic should probably patch class-pclzip.php to deal with gzopen/gzopen64 as there are a great many broken PHP installs out there (the PHP bug has been open for over a year and a half without what I would consider an acceptable solution), and it is surprisingly difficult to find a solution online. I guess a great many WordPress installs are still 32-bit, which is kind of sad.

Changing the WordPress table prefix

This may be of use to people experiencing the dreaded “You do not have sufficient permissions to access this page.” message when trying to reach WordPress’ admin page, even when logging in as a proper administrator. WordPress embeds the table prefix in 4 different locations:

  • The wp-config.php file
  • The name of the tables
  • The name of user metadata keys
  • The name of blog options

Thus if you want to change the prefix, you have to:

  1. Edit wp-config.php to change the prefix
  2. Rename your tables
  3. Rename your user metadata
  4. Rename your site options

Missing steps 1 or 2 will cause WordPress to not find the tables, and it will go through the initial install process again.

Missing step 3 will cause the account to lose its roles, and thus not be authorized to do much besides read public posts.

Missing step 4 is more insidious, as it destroys the option wp_user_roles, the link between roles and capabilities, and thus even if your account is an administrator, it is no longer authorized for anything.

It feels quite clunky to embed the database prefix in column values, not just tables, just like WordPress’ insistence on converting relative links to absolute links. The former makes moving tables around (e.g. when consolidating multiple blogs on a single MySQL database) harder than necessary. The latter makes moving a blog around in a site’s URL hierarchy break internal links. I suppose there are security reasons underlying Automattic’s design choice, but security by obscurity of the WordPress table prefix is hardly a foolproof measure.

If you are renaming the tables, say, from the default prefix wp_ to foo_, the MySQL commands necessary for steps 2–4 would be the following:

UPDATE wp_usermeta SET meta_key=REPLACE(meta_key, 'wp_', 'foo_')
WHERE meta_key LIKE 'wp_%';
UPDATE wp_options SET option_name=REPLACE(option_name, 'wp_', 'foo_')
WHERE option_name LIKE 'wp_%';
ALTER TABLE wp_commentmeta RENAME TO foo_commentmeta;
ALTER TABLE wp_comments RENAME TO foo_comments;
ALTER TABLE wp_links RENAME TO foo_links;
ALTER TABLE wp_options RENAME TO foo_options;
ALTER TABLE wp_postmeta RENAME TO foo_postmeta;
ALTER TABLE wp_posts RENAME TO foo_posts;
ALTER TABLE wp_redirection_groups RENAME TO foo_redirection_groups;
ALTER TABLE wp_redirection_items RENAME TO foo_redirection_items;
ALTER TABLE wp_redirection_logs RENAME TO foo_redirection_logs;
ALTER TABLE wp_redirection_modules RENAME TO foo_redirection_modules;
ALTER TABLE wp_term_relationships RENAME TO foo_term_relationships;
ALTER TABLE wp_term_taxonomy RENAME TO foo_term_taxonomy;
ALTER TABLE wp_terms RENAME TO foo_terms;
ALTER TABLE wp_usermeta RENAME TO foo_usermeta;
ALTER TABLE wp_users RENAME TO foo_users;

Incensed at Mozilla

One of the greatest features in the Webkit-based browsers (Apple’s Safari and Google Chrome) is WebSQLdatabase, the ability for a web site to store information in a SQLite database on your browser accessible via JavaScript. This allows web developers to build database-enabled applications that run entirely in the browser, without requiring a server. This is very useful for mobile devices, which in the US enjoy flaky network connectivity at best. One very handsome example is the iPad-optimized Every Time Zone webapp.

SQLite is probably the most important open-source project you have never heard of. It is a simple, streamlined and efficient embedded database. Firefox stores its bookmarks in it. Google distributes its database of phishing sites in that format. Sun’s industrial-strength Solaris operating system stores the list of services it runs on boot in it—if it were to fail, a server would be crippled so that is a pretty strong vote of confidence. Adobe Lightroom and Apple’s Aperture use it to store their database, as do most Mac applications that use the CoreData framework, and many iPhone apps. In other words, it is robust and proven mission-critical software that is widely yet invisibly deployed.

WebSQLdatabase basically makes the power of SQLite available to web developers trying to build apps that work offline, specially on mobile devices. No good deed goes unpunished, and the Mozilla foundation teamed up with unlikely bedfellow Microsoft to torpedo formal adoption of WebSQLdatabase as a web standard, on spurious grounds, and pushed an alternate standard called IndexedDB instead. To quote the Chromium team:

Q: Why this over WebSQLDatabase?

A: Microsoft and Mozilla have made it very clear they will not implement SQL in the browser.  If you want to argue this is silly, talk to them, not me.

IndexedDB is several steps backwards. Instead of using powerful, expressive and mature SQL technology, it uses a verbose JavaScript B-tree API that is a throwback to the 1960s bad old days of hierarchical databases and ISAM, requires a lot more work from the developer, for no good reason. To add injury to insult, Firefox 4’s implementation of IndexedDB is actually built on top of SQLite. The end result will be that web developers will need to build a SQL emulation library on top of IndexedDB to restore the SQLite functionality deliberately crippled by IndexedDB. If there is one constant in software engineering, it is that multiple layers add brittleness and impair performance.

Of course, both Mozilla and Microsoft are irrelevant on mobiles, where WebKit has essentially won the day, so why should this matter? Microsoft has always been a hindrance to the development of the web, since they have to protect the Windows API from competition by increasingly capable webapps, but I cannot understand Mozilla’s attitude, except possibly knee-jerk not-invented-here syndrome and petulance at being upstaged by WebKit. WebSQLdatabase is not perfect—to reach its full potential, it needs and automatic replication and sync facility between the local database and the website’s own database, but it is light years ahead of IndexedDB in terms of power and productivity.

I am so irritated by Mozilla’s attitude that after 10 years of using Mozilla-based browsers, I switched today from Firefox to Chrome as my primary browser. Migrating was surprisingly easy. Key functionality like bookmark keywords, AdBlock, FlashBlock, a developer console, and the ability to whitelist domains for cookies, all have equivalents on Chrome. The main regressions are bookmark tags, and Chrome’s sync options are not yet equivalent to Weave‘s. At some point I will need to roll my own password syncing facility (Chrome stores its passwords in the OS X keychain, which is also used by Safari and Camino).

RapidSSL 1 – GoDaddy 0

My new company’s website uses SSL. I ordered an “extended validation” certificate from GoDaddy, instead of my usual CA, RapidSSL/GeoTrust, because GoDaddy’s EV certificates were cheap. EV certificates are security theater more than anything else, I probably should not have bothered.

Immediately after switching from my earlier “snake oil” self-signed test certificate to the production certificate, I saw SSL errors on Google Chrome for Mac and Safari for Mac, i.e. the two browsers that use OS X’s built-in crypto and certificate store. I suppose I should have tested the certificate on another server before going live, but I trusted GoDaddy (they are my DNS registrars, and competent, if garish).

Big mistake.

I called their tech support hotline, which is incredibly grating because of the verbose phone tree that keeps trying to push add-ons (I guess it is consistent with the monstrosity that is their home page).

After a while, I got a first-level tech. He asked whether I saw the certificate error on Google Chrome for Windows. At that point, I was irate enough to use a four-letter word. Our customers are Android mobile app developers. A significant chunk of them use Macs, and almost none (less than 5%) use IE, so know-nothing “All the world is IE” demographics are not exactly applicable.

After about half an hour of getting the run-around and escalating to level 2, with my business partner Michael getting progressively more anxious in the background, the level 1 CSR tells me the level 2 one can’t reproduce the problem (I reproduced it on three different Macs in two different locations). I gave them an ultimatum: fix it within 10 minutes or I would switch. At this point, the L1 CSR told me he had exhausted all his options, but I could call their “RA” department, and offered to switch me. Inevitably, the call transfer failed.

I dialed their SSL number, and in parallel started the certificate application process on RapidSSL. They offered a free competitive upgrade, I tried it, and within 3 minutes I had my fresh new, and functional certificate, valid for 3 years, all for free and in less time than it takes to listen to GoDaddy’s obnoxious phone tree (all about “we pride ourselves in customer service” and other Orwellian corporate babble).

I then called GoDaddy’s billing department to get a refund. Surprisingly, the process was very fast and smooth. I guess it is well-trod.

The moral of the story: GoDaddy—bad. RapidSSL—good.

Update (2012-08-26)

I switched my DNS business from GoDaddy to Gandi.net in December 2011 after Bob Parsons’ despicable elephant-hunting stunt.

Scientific papers now citing blogs?

In my misspent youth I spent about a year as a visiting scholar researching wavelets under Raphy Coifman’s supervision at Yale’s small but excellent Mathematics Department. Professor Coifman was head of a department that also featured Benoît Mandelbrot (of fractals fame), the late Walter Feit (as in the Feit-Thompson theorem), and Fields medalist Gregory Margulis. He was kind enough to credit me on a published paper, even though he did all the work, reverse delegation in action. That paper had modest success and was cited, so I can claim (not with a straight face) to be a published mathematician.

While perusing my blog’s web analytics referrer report, I was surprised to find out my article on the Nikon D70’s not-so-raw RAW format is actually cited in a serious scientific paper on human vision. We keep hearing about students getting flunking grades for citing Wikipedia, are blogs really considered more authoritative?

The citation uses the old URL for the blog entry, </mylos/weblog/2004/05/02-1.html>. When I migrated to WordPress at the end of 2009, I took great pains to provide redirects whenever possible and avoid broken links. Many bloggers don’t have the time or expertise to do this, and simply leave dangling permalinks around. If quoting blogs is to become standard practice, authors should probably provide some sort of fallback mechanism like linking to Archive.org, but dead-tree journals do not have this capability. Absent that, linkrot may spread to an entire new category of documents, scientific papers.

Update (2011-07-18):

Here’s another paper (PDF) referencing the same article. What’s next, CiteSeer?