There are now 10,000 Android developers, as opposed to 40,000 for the iPhone. And Google is gearing up to release App Inventor, a tool for nontechnical Android app developers that is strikingly similar to the Scratch programming environment for kids. Meanwhile Android phones themselves are catching up quickly, although the iPhone 4 remains the coolest thing around.

Like many others I’m sure, I considered developing a tool like this and abandoned it after coming to grips with Apple’s harsh policies on the subject of alternative programming environments:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

Google’s reliance on crowdsourced quality control for apps isn’t perfect but it is beginning to add up to a real difference in spirit. As Android phone sales grow, making development for it an even more viable proposition, content creators will eventually vote with their feet for the platform where they feel most welcome and walk away from the walled garden.

I have an iPhone and I love it, in part because of Apple’s excellent quality control in the core applications. If Apple’s competition here were as bad as their competition in the MP3 player market— companies with almost no grasp of design— I wouldn’t doubt the outcome for a moment. But Android also has quality designers at play. Google’s approach to design may be a bit plodding and obsessed with A/B testing, but it does get results.

Apple knows they must continue to release cutting-edge hardware, but they don’t seem to grasp what it takes to maintain a welcoming developer platform once there are viable alternatives in the game.

I’m going out on a limb here: a year from now there will be as many Android apps as iPhone apps. 18 months from now Android phone sales will surpass iPhone sales. Apple will still have the sexiest phones, and will still command a premium price for them. But inside two years the iPhone will occupy a niche similar to that of the Macbook.

WHAT: Symfony 2 Live Conference
WHERE: P’unk Avenue offices, 1168 E. Passyunk Ave, Philadelphia PA
WHEN: 11am Wednesday, June 23rd

Thanks to our hub status, You do not have to buy your own Symfony 2 ticket to attend at our office.

We’re looking forward to discussion and possibly busting out the Symfony 2 sandbox and experimenting after the conference.

No, not by a long shot, not any time soon. Symfony 2 will not be backwards compatible in a “super easy upgrade path” way, which is a valid choice to make improvements that would otherwise be impossible but an obvious problem for existing projects. And we have plenty of those. What’s more, Symfony 2 requires PHP 5.3.x at a minimum, and we know how tough it can be to move clients along just to PHP 5.2.x.

Fortunately, Sensio had the wisdom to make Symfony 1.4 a long-term-support release. And since Apostrophe 1.x is happily married to Symfony 1.4, that means we’re all in good shape.

Symfony 1.4 is supported until November of 2012, and Apostrophe 1.x will also be supported until at least November of 2012.

At some point, Apostrophe 2.x will come into being and it will be based on Symfony 2, but we have many client and internal projects that are purring away happily on Symfony 1.4 and we have absolutely no interest in abandoning it. So you can very safely pursue client projects based on the current stable releases of Apostrophe.

Speaking of Symfony and being busy, if this announcement disappoints you— if you’re eager to hobnob with other Symfony developers— may I suggest you apply to work here? You’ll have all the Symfony you can eat! We’re particularly eager to talk to experienced PHP5 framework coders, and to Ruby on Rails coders who are open to working in PHP (sekrit plans to convert us to Ruby are a bit unrealistic given the depth of our commitment to PHP and Symfony).

Things are moving pretty fast lately on our corner of E. Passyunk and Federal. Apostrophe is rolling along at a full clip and we have all sorts of other exciting projects we’re in the midst of. Olivia Rabe recently joined the team to work closely with our clients on issues of content strategy and Apostrophe support. We’re even beginning to expand our studio space into the adjacent building.

If this sounds at all interesting to you, the best news is that you might be able to help us fill some of our new space: We’re looking to make another addition to the team so we can continue to build these amazing applications and still go home on time. Right now we are looking for a new developer, somebody who would assist in the creation of new Apostrophe features and collaborate on development of products for our clients.

If you think you might be the right fit, or know someone who is, read on after the cut.

P’unk Avenue is a web design and development team located in Philadelphia that creates websites and applications for higher education and business clients. We are also the creators of Apostrophe, an open source CMS built on top of the Symfony framework.

We are eight people looking for another developer to add our team. Since we are a small shop we all get involved in various aspects of each project. We believe that is a good thing and are looking for someone that enjoys being engaged in a project on many levels. Your role would involve a high level of collaboration with the entire team as you help us see projects through from designing and prototyping to development and deployment.

Our ideal colleague meets several of these criteria:

Fluency and comfort with PHP & MySQL
A deep and practical understanding of MVC frameworks (Ruby on Rails, Zend Framework, CakePHP, Django and CodeIgniter all apply, but experience with Symfony is especially desirable)
A working knowledge of ORMs such as Doctrine, Propel or ActiveRecord
Gets excited about things like web services, rapid prototyping and open source projects
A passion for using Test Driven Development to write tight, well tested code
A love for collaborating with a small team to solve tough problems
A sense of humor and an ability to stay cool under pressure

This job does require you to work out of our (awesome) Philadelphia studio. We have great flexibility in other areas, but we believe that having our team in the same location on a regular basis benefits the products that we create.

Of note, we organize various community events like the Junto and Ignite Philly. We are very interested in engaging in and encouraging deeper conversations about stimulating ideas. The type of person that is also stimulated by thinking and acting on issues they care about would be a good fit on our team. It is not required, but the opportunity to help organize these events is a possibility for any member of our team.

Interested?

If you feel that you are a good fit, please send resumes and whatever other documentation that you think best represents why we should consider you to: jobs@punkave.com.

You may recall my recent post on speeding up PHP on Linux without changing your PHP code. Those techniques have become standard practice around here, and the article has since been retweeted, linked to, engraved on an unobtanium disc embedded in a space probe to be launched next week by the Malaysian space forces, et cetera. So it seems we have the Linux PHP thing down to a science, for now.

But not every shop runs Linux, and one of our clients has asked us to deploy in a Windows environment. We’ve deployed PHP sites for them before, but that was in the Symfony 1.0 days, and there just weren’t nearly as many class files to compile. When we set out to deploy a modern site built with Apostrophe, our content management system based on Symfony 1.4 and Doctrine, we were reminded that out-of-the-box PHP is slow, slow, slow.

I would gladly have addressed this the same way we do it on Linux: FastCGI, APC and Apache’s worker thread MPM. And in fact Apache for Windows seems to only support a special threaded MPM under Windows anyway, so that part is already familiar.

Unfortunately, the official PHP infrastructure for Windows has, frankly, undergone a bit of code rot lately:

The PHP binaries don’t seem to include a CGI/FastCGI binary! Shoot. Well, maybe APC alone will be enough. But APC is not standard with the Windows PHP binaries, so I need to get it from PECL. Okay, but the PECL for Windows site is “temporarily out of service.” A new solution is coming, but it’s not here yet.

Searches led me to Pierre Joye’s folder on downloads.php.net. Pierre is a core PHP developer, so I trust him, no problem there. But the version of APC found there just locks up Windows PHP, at least in my tests. I investigated alternative PHP bytecode caches and wound up with xcache. xcache worked for a while, then started spewing errors about classes being redefined, and always began doing so again within a few page views after each restart of Apache.

Yuck!

Fortunately, by this point a vague memory was stirring somewhere in my geeky cortex. I was in Paris… I was full of pain au chocolat… I was very sleepy… two guys in suits were droning on and on about something while the crowd ignored them… AHA! Those Microsoft guys at Symfony Live! They were talking about speeding up PHP on Windows. They even mentioned hiring Pierre Joye to help out. But they didn’t demonstrate anything on Symfony, so I wasn’t paying a lot of attention—

Just enough as it turns out. The deal is this: Microsoft knows that PHP has, historically, sucked on Windows. And they have taken steps. And this is good news for the rest of us.

PHP can now be installed via the Microsoft Web Platform installer. That gives you single-threaded PHP running via fastcgi— exactly like my Linux setup, except for the bytecode cache part. And now the new Windows Cache Extension for PHP delivers a bytecode cache, equivalent to APC.

The end result is definitely in the ballpark with the performance of my Linux solution. Since I haven’t tested them on the same hardware (both boxes were VMs, and they were provided by different datacenters), I cannot say for certain which was faster. The Microsoft speakers at Symfony Live characterized it as better but still not as fast as the best PHP setups on Apache. At any rate it’s a big improvement and well worth your time if you need to host on Windows.

So here’s a HOWTO for configuring Symfony and PHP on Windows with good performance:

To configure IIS for Symfony:

1. Make sure IIS is installed. I know, right?

We want IIS 7. IIS 7 features an excellent importer for mod_rewrite rules, and Symfony loves mod_rewrite rules.

2. Install PHP from the Microsoft Web Platform. Click that link from IE running on your server (hint: remote desktop connection) for the easiest download-and-install process.

3. Install Windows Cache for PHP to get acceptable speed from the above. I installed version 1.0 because 1.1 is in beta. The beta does look interesting because it has an alternate PHP session handler that uses shared memory. You’d have to configure symfony to actually use that feature I believe (welcome to factories.yml).

4. Let’s say your project’s short name (aka “Unix name”) is mysite. So copy your Symfony project to c:\mysite. (I’m assuming a virtual machine for a single client project. Choose a better folder structure if you’re deploying lots of sites on this box.) You can set up cygwin to allow ssh access and rsync those files with project:deploy, and in fact we do, but that’s a subject for another day. FTP also works.

5. Launch the IIS manager (Start -> Administrative Tools -> IIS Manager).

6. Open the server’s icon, then “Sites,” then delete the “default site” entry.

7. Add a new site entry called mysite. Map the site’s web folder to be the web subfolder of the Symfony project which you should have copied to c:\mysite by now. So the web folder path would be c:\mysite\web. Assign it to port 80. Do NOT set the web folder to be the root of your project! That is very dangerous as it exposes your databases.yml file. Point it to the web subfolder.

8. Always clear your Symfony cache after copying a site to a new server. At the command shell (start->run->cmd.exe), type:

cd \mysite
php symfony cc

10. PHP needs write access to c:\windows\temp to store session data in order for logon sessions to work. You should have no trouble here on a new setup with just IIS I suspect, but if you previously ran Apache you might need to clean up. Clobber any existing session files in c:\windows\temp. Note that this is also the folder where the PHP error log file is found:

c:\windows\temp\php-errors.log

You can watch that file using a Cygwin shell and the tail -f command, or peek at it as needed with a text editor.

9. Try the site out, with a frontend controller name in the URL. You’ll get the home page but subpages won’t work yet without manually putting the controller name back in the URL. In IE running on the server itself, visit:

http://localhost/index.php

10. We need to add the mod_rewrite rules that remove the need for `/index.php`. But IIS doesn’t have mod_rewrite. Fortunately IIS7 has its own URL rewriting feature… and it can import your rules.

In the IIS manager, double-click mysite, then double-click “URL Rewrite.” In the “Actions” area at right, click “Import Rules” under “Inbound Rules.”

Edit: if “Inbound Rules” is missing, close “URL Rewrite” and then open that feature again. Also try adding one empty rule manually first. The “incredible disappearing/reappearing ‘Inbound Rules’ feature” appears to be a bug in the IIS manager.

Paste the mod_rewrite rules from your web/.htaccess file. IIS will do a pretty snazzy job of converting them when you click Import (but if you have fancy rules that go beyond the usual Symfony rules, give the end results a careful look-see). Then click “Apply.”

Edit: IIS won’t tell you, but the settings you just created live in a file called web.config in the web subfolder of your project. You must not crush this file when you sync your project. Add the file to your project, or rsync exclude it if you’re using cygwin and project:deploy. Since you might want to add more rules to it manually I generally recommend the former. But one disadvantage of adding it to your project is that any edits you make in the GUI will get clobbered on future syncs if you don’t manually copy the resulting changes from web.config on the server back to your project.

11. Try the site again. You should be able to browse the entire site now.

12. Whoops, you can’t log in because IIS is hijacking the 401 Unauthorized error page. To fix that, go back to the IIS Manager pane for mysite. Double-click “Modules” and remove the “Custom Error” module.

Now your site’s login page will appear properly. Also your custom 404 and 500 error pages.

That’s it – you should be ready to go with much speedier Windows PHP.

Note that for the time being the Microsoft Web Platform seems to offer only PHP 5.2.13. This is a stable and sane choice and until recently I would have done it no differently. But PHP 5.3.2 is much better than 5.3.0 and 5.3.1 and offers new features that matter— improved memory management, faster performance and new PHP language features that the forthcoming Symfony 2.0 requires. Hopefully PHP 5.3.x will appear in the Web Platform installer soon.

(*) Yes, I sorted through the maze and found versions of PHP and APC that were supposedly for the same C runtime, et cetera. Still no love. xcache was better, but alas the redefined class errors put paid to that too.

CALLER: “I chose your open source project for my client project and it’s due tomorrow and it doesn’t work! This is your problem and you must fix it immediately or I will explode!”

TOM: [CLICK]

CALLER: “Your code has several bugs, which I am going to describe in harsh and unsympathetic but highly accurate detail.”

TOM: [Hangs up on caller, then fixes those bugs]

CALLER: “I translated your program into Swahili! Here’s the XLIFF file. Also I found a bug. Here’s a patch file.”

TOM: “Yay! Welcome to the cool people’s club! We’re having chocolate ice cream. Would you like some?”

CALLER: “500 ERROR! AAAAAAAAAAAAAA!”

TOM: “Lower your voice, read your log files, try the dev controller and call back.”

CALLER: [reads own log files, discovers glaringly obvious permissions problem, solves it, is never heard from again]

CALLER: “something isn’t quite right. When I do X, Y and Z, I get the following error message [copies and pastes complete error message].”

TOM: “reproducible bug reports make me deeply happy. I’ve just committed a fix in svn. Would you like some ice cream?”

Oh yeah: the Philly Symfony Users Group. We use the Symfony framework for PHP development all day long. It’s been the basis for every single client project we’ve done since… hell, before I came aboard two years ago. It’s the foundation on which Apostrophe is based. We cannot imagine suffering through web application development without it (or at least something on a similar plane of object-oriented, thoroughly modern, Model-View-Controller awesome, like Zend Framework).

But we haven’t met a lot of other Philly-based Symfony developers. And this is where you come in. We’d like to spread the word and help aspiring local Symfony developers get their boots on the ground. And if other experienced Symfony developers are out there reading this, we’d really, really like to get to know you better.

Announcing the Philly Symfony Users Group! Every second Thursday of the month at 6:30pm, you are cordially invited to our office at 1168 East Passyunk Avenue. We’ll start off with informal discussion (and beer), then someone will present on an aspect of Symfony development.

WHERE: 1168 E. Passyunk Ave
WHEN: 6:30pm, Thursday April 8th
WHAT: Symfony Users Group
PRESENTATION: “Getting Started with Symfony,” Tom Boutell

At this first meeting I’ll talk about the process of installing Symfony, PHP and Apache on your own computer, focusing primarily on what it’s like to build a simple Symfony 1.4 web application.

You don’t have to be an experienced Symfony developer to attend! We welcome all PHP developers and those who aspire to become PHP coders.

If you visit our demo site and switch languages you’ll see pages with pretty translated chrome but no content. That, of course, is because our own demonstration web site’s content is currently only available in English. Feel free to play with translating it, but keep in mind that the demo site resets at the top of the hour. If you’re seriously interested in translating our demo site’s content to go along with the nicely translated interface, join the Apostrophe google group and drop us a line to get involved.

Interested in putting this power to work on your projects? Visit apostrophenow.com to find out how we can help you. If you’re a Symfony developer or just a ballsy webmaster who doesn’t mind learning some new tricks, head straight for the Apostrophe manual to check out our installation guide. We do documentation in a big way.

We’d like to thank Quentin Dugauthier (French), Frank Wenzl (German), and Pablo Godel (Spanish) for their hard volunteer work on the translations. Others are working hard on Portuguese and Greek and have been patient through our growing pains as we mastered the fine points of UTF-8 and character sets in our translation system. We’ll be sure to thank them by name when they are done, so you don’t nag them too much now. (:

There are, of course, other important fixes and improvements in Apostrophe 1.0.9, including one security fix. Here’s the list:

• A computer abandoned by an admin who has logged out can no longer be used to edit slots the admin previously edited using cleverly constructed URLs (only an issue on the same computer and if the PHP session has not ended). Note that you must upgrade your myUser class in apps/frontend/lib to extend aSecurityUser rather than sfGuardSecurityUser to get this fix (aSecurityUser is a subclass of the latter)
• Global or virtual-page media slots can be edited successfully on Symfony pages that are not CMS pages
• Unpublished pages no longer interfere with aNavigationAccordion layout
• Fixtures no longer use HTML tags our filters remove on edit
• Plaintext slots now autolink URLs and email address (obfuscated) as described in the manual
• Search engine updates refactored, search engine now updates when you save page settings
• ‘tool’ option to rich text slots now correctly activates the FCK toolbar set name you specify
• Slot save/cancel buttons now survive form validation passes properly (thanks to Spike)
• Date widget is XHTML correct (thanks Spike)
• Engines now work when the CMS is not mounted at the root of the site (important for those using the CMS as a subfolder of a site dominated by other Symfony modules)
• Attempting to attach a list of zero items to a slideshow no longer results in adding all items in the media repository
• Cross-browser and XHTML strictness fixes
• Moved lib/base to lib/action (you must symfony cc)
• Lost connections between existing media slots and media items when editing other media slots: fixed. Also, slideshows etc. are no longer removed on “cancel,” and selecting zero media items no longer selects all media items
• i18n of over 99% of the admin interface (many thanks to Quentin, Galileo, Frank, Pablo and Fotis), new languages are regularly being added to the demo project’s apps/frontend/i18n folder
• More convenient i18n of your site content (temporary titles supplied, all navigation controls work for pages whose titles are not yet translated)
• Aesthetic upgrades
• Superadmins can grant superadmin status
• Some demo-specific styles moved from a.css to demo.css
• Optional language selector in a/login partial
• Global admin buttons now have separate names and labels (labels can be internationalized) and a documented way to add and reorder them in app.yml
• Alpha channel is now preserved when rendering PNGs from a PNG original with gd (not available with netpbm)
• Compact PDF slot style, without inline preview (you can override this in aMediaPDF/normalView if you want it back and you have ghostscript)
• Better IE6 upgrade message
• Various private methods now protected for easier app level overrides

With frameworks like Symfony, you can build them faster still, and follow modern programming practices at the same time.

And Apostrophe strips away yet another layer of effort if your site calls for a content management system.

Yes, Java has more raw speed, all else being equal (which it never is). But as the LISP programmers used to say, “a moment of regret, a lifetime of convenience.”

Still, sooner or later success catches up with you and you want your site to cope with Serious Traffic… or cope with moderate traffic on a cheap virtual machine… or at the very least, not be dog-slow with just a handful of users on the system.

There’s a lot of advice out there about optimizing PHP code, some of it well worth your while. And there’s excitement about HipHop, Facebook’s new native code compiler for PHP. But these are drastic steps that require you to rewrite your code or adopt less proven and more awkward ways of delivering your code.

Justified? Sure, sometimes, on the biggest projects in the world (like Facebook) (*). But as Donald Knuth says, “premature optimization is the root of all evil.” That’s because tweaking your code for speed’s sake usually makes it harder to maintain and less adaptable to new requirements.

What most developers don’t realize is that there are three major factors that typically slow down PHP projects based on frameworks (like Symfony or, sigh, Drupal) so much that code profiling and database query redesign don’t even have a chance to become relevant factors. Fix these things first before you worry about other issues:

1. Compiling code over and over and over. Would you wait for your Mac to recompile MacOS X from source code every time you boot it up? Of course not. How about every time you fill out a dialog box? That’s pretty much what you’re doing every time you access a PHP-driven website that doesn’t use a bytecode cache.

2. Waiting and waiting and waiting for web browsers to make another request, pinning down web server processes that your other users need. By default Apache usually lets browsers hold on to a connection for up to 15 seconds just in case they ask for more. This is a good thing in many ways, but 15 seconds is far too long. Which leads us to #3:

3. Tying up a “fat” web server process with PHP on board for every request, even requests for the zillions of little static PNGs that probably make up your page design. (**) A typical Apache web server configuration with mod_php suffers from this flaw, fatally limiting the number of simultaneous users you can handle.

So what can we do about these problems? Quite a bit as it turns out. I’ll start with the low-hanging fruit and move on to the tougher stuff. The fascinating common thread with all of these suggestions: no changes at all to your PHP code.

Stop Recompiling Your World: APC

APC, aka the Alternative PHP Cache, is a bytecode cache. What does that mean and what good does it do?

Today, when users visit your website, your web server struggles to translate all of your PHP code into a more manageable “bytecode.” And only then does it actually run the bytecode and output the page.

Bytecode isn’t true native machine code and doesn’t run as fast as compiled Java, and certainly not as fast as compiled C. That’s why Facebook’s HipHop project seeks to convert PHP code to C++, and then compile that to native code. But that’s a drastic step with far-reaching consequences for your project. Before you think about HipHop, try a simpler fix: stop throwing away the bytecode and starting over from zero on every new request that comes to the site!

In our experience, enabling APC reduces both memory usage and the time required to respond to a request by vast amounts. Even a simple action in Symfony or Drupal can implicitly require that quite a bit of PHP code be compiled. With APC, all of that code stays compiled, and that simple action actually runs quickly and simply.

APC is simply indispensable for serious PHP programming. Turn it on. Make sure it’s really working. And enjoy dramatic improvements in your social life, hair and skin.

Installing APC

Depending on your system, APC may already be installed or even, if you are very lucky, activated correctly. Virtual machine hosting from ServerGrove comes with APC “out of the box.” Most shared hosting probably won’t because it increases the amount of memory pinned down for each user on the shared server, but if you care about your website’s speed, you’re not using shared hosting. Get a small, cheap virtual machine from ServerGrove instead (or if you must, do it the hard way and set up a Slicehost or linode virtual machine; you’ll learn a lot about system administration).

On our slicehost VMs, we found it necessary to install APC with pecl, the PHP package manager for extensions (like PEAR, but different in that they are not written in plain PHP):

pecl install apc

We then enable it in php.ini (you won’t be able to do this in a .htaccess file):

; Must enable APC otherwise we compile all of Symfony on every hit
extension=apc.so
; Give APC enough shared memory to handle Symfony, etc.
apc.shm_size = 48
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

Then restart Apache:

apachectl restart

And you’re off to the races. The first page access will be as slow as ever, but subsequent accesses will be much, much faster. You can also check whether you’re saving memory and time using Symfony’s debugging toolbar.

Your system also likely has an APC status page that you can copy to your website to check out the current status of APC. It’s fun to play with.

Dialing Back Keepalive: Too Much of a Good Thing

In the beginning there was NCSA Mosaic, and it was pretty cool, all things considered. But it wasn’t fast.

Mosaic made only one request of a web server at a time. Trying to do more probably seemed rude. Besides, lots of web servers couldn’t cope with more than one request a time at all. But that would only happen if two users came along at once, and that was never gonna happen, right?

Ah, those were the (sad, lame, pathetic) days. Scroll to the present. Typical web browsers first request the page, say “hang on a minute,” open perhaps five more connections besides and then make note of all the images, JavaScript files, CSS files and other embedded bric-a-brac and start requesting all of those things. Consecutively and in parallel. Really a lot.

The Apache web server is actually very good at handling this. But by default, it is very generous: it lets the browser leave it hangin’ for fifteen seconds. That means an Apache process and all of the memory it requires is tied up for all of that time, just doin’ nothin’. And that’s bad news.

Fortunately it’s very easy to fix this in your httpd.conf file (actually, in our setups, this is usually in apache2.conf):

KeepAliveTimeout 2

Restart Apache:

apachectl restart

And you’ll find that your site can suddenly cope with higher loads. That’s because your’re not tying up valuable resources waiting around for more requests from browsers that are already using and reusing five other connections to your server. The browser is welcome to promptly request more files and finish its business with you, but not to stand around twiddling its thumbs while you wonder if you have a date or not.

You can turn keepalive off entirely, but doing so forces the browser to “call back” for each and every little image file on your page, which is a terrible idea. So use it… just don’t let it abuse you.

MaxClients: Does This mod_php Make My Butt Look Big?

mod_php is fast. Unlike the old-fashioned “PHP as CGI” approach, mod_php keeps the PHP interpreter itself loaded in memory and ready to rock at all times, and that’s good.

But mod_php is also a pig. For a big-deal web application that wrangles lots of database objects, or makes thumbnails of uploaded images, or does pretty much anything cool, it’s not uncommon for a PHP process to use 50MB of memory or more. More miserly code can still hit 30MB without much difficulty.

We first ran into this problem when we wanted to move smaller client sites onto small VMs, so that they could have well-secured, highly-reliable web hosting at an affordable price. They tended to bog down and become unusable quickly.

We discovered that dialing back keepalive helped tremendously with the problem. And we also learned to do the math and figure out how many reasonably fat PHP Apache processes would really fit in the memory available to the slice, and configure Apache to run no more than that. In httpd.conf:

MaxClients          20

This may seem drastic (OMG! I can’t serve more than 20 simultaneous users!) but here’s the thing: you already can’t. If you try, if you leave this set to its much higher default, the result will be a VM crushed by the need to swap out all of these Apache processes to virtual memory while they are still trying to do work. As the operating systems geeks say, the “resident set size” of your Apache and PHP processes is bigger than your actual memory. And when that happens you’re SOL.

When you set MaxClients to a realistic number, what happens instead is this:

You can handle up to 20 connections at once gracefully.
That 21st connection queues up and is dealt with soon after.
If the average load doesn’t exceed 20 connections, you’re fine.

If too many connections queue up, some people will not get through. But you will still be able to get through to other services on the VM… like the ssh connection you use to administer it and change these settings. Without that, you’re stuck in a seesaw battle, shutting down your website, tweaking settings, firing it up again and watching it gradually become overwhelmed.

What Is The Right MaxClients For Me?

Take the amount of RAM in your server (for instance, a 256MB slicehost VM). Subtract a bit of overhead (at least 50MB, ideally more, bigger VMs are better that way). Now divide by the largest PHP process you typically see when running “top” or “ps -auxw | grep httpd” (you might have to grep for apache rather than httpd depending on your OS). For instance, if your processes top off around 30MB, you can set MaxClients to (256-30) divided by 30. Round down and you have a MaxClients setting of 7.

One snag: some operating systems show shared memory (like the memory used by APC) as memory in use by every one of the Apache processes, even though it is really only allocated once and shared by all of them. If your APC cache is set to 40MB and all of your PHP processes seem 40MB overweight, you can increase MaxClients somewhat to allow for this.

Yes, you can use swap space (virtual memory) to go higher, but this just forces the machine to swap, using disk space in place of memory to do hard things it needs to do quickly and freezing you out of your own machine. So don’t do that.

PHP, I Love You, But Could You Sleep Downstairs?

Dialing back the number of Apache processes is a helpful step that allows your server to say no with grace and style when there is too much traffic. But the underlying problem remains: we can’t deal with a lot of requests at once. And the reason we can’t do that is PHP. So we’re stuck. Or are we?

Our Apache processes are “fat” in terms of memory use because of the memory taken up by PHP. But most of the requests we’re handling are for little PNG files and not-so-little JPEG files and CSS files and JavaScript files… all of which Apache can serve up handily with about three bytes of memory and a piece of string.

The obvious solution is to serve PHP requests and static files separately. And many sites do this explicitly, using absolute URLs pointing at a different subdomain for static files. For example:

<img src=”http://static.example.com/foo.png” />

Now, this is a fine thing and I have nothing against it. When you design things this way you can use entirely separate machines to serve these requests if you wish, and that’s pretty exciting from a performance perspective. Many bigger sites find this well worth their while.

But you are also complicating things. And you are forced to change working code, which introduces the potential for new and exciting bugs.

There’s another approach (well, one of many): FastCGI. FastCGI is an enhanced version of the CGI protocol that appeared quite early in the evolution of the web but caught on only with those most concerned about performance. I must admit I am a very recent convert, ironic since FastCGI is a bit out of vogue today. But there are solid reasons to consider using it.

FastCGI has three big advantages:

1. FastCGI runs PHP processes separately from Apache. That means Apache can stay lean and mean, not only running much smaller processes (and therefore perhaps many more of them) but even running in a separate “worker thread” that outperforms the separate-processes mode by a long mile. This is a big, big win.

2. It can reuse a single process for new PHP requests (***), avoiding the need to restart the PHP interpreter. That’s nice, but mod_php can do the same trick, and mod_php is capable of sharing APC cache memory between all of its processes. And FastCGI can’t do that… or it wouldn’t be able to without a little help from PHP:

3. PHP comes with a special FastCGI mode in which it manages several child processes of its own, and allows all of those processes to share a single APC cache.

Put these advantages together and you have the holy grail of PHP hosting: PHP running separately from Apache, but still maintaining all of the performance advantages of the APC cache. And we can do it without changing a line of PHP.

Installing FastCGI

There’s one catch: Apache 2.0 ships with mod_cgid, a rewritten version of FastCGI that complies with the official Apache license, instead of the oh-so-fractionally-different FastCGI license (****). And it’s nice and all, except that it completely lacks the third advantage of FastCGI I mentioned above. If you want the benefits I’m describing here, you have to use the real mod_fastcgi. Accept no substitutes.

These directions work for slicehost Ubuntu slices and other Ubuntu-based Linux. They will not work exactly as-is for servergrove which is centos-based:

Edit /etc/apt/sources.list
Add multiverse, which includes software with insufficiently “open” licenses, such as mod_fastcgi. Change intrepid to the ubuntu version on your box, it’ll be mentioned all over sources.list:

# tom@punkave.com: we want multiverse support so we can have real mod_fastcgi
deb http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
deb http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse

Now do:

apt-get update
apt-get install libapache2-mod-fastcgi

Now edit /etc/apache2/mods-available/fastcgi.conf to read:

<IfModule mod_fastcgi.c>
  # One shared PHP-managed fastcgi for all sites
  Alias /fcgi /var/local/fcgi
  # IMPORTANT: without this we get more than one instance
  # of our wrapper, which itself spawns 20 PHP processes, so
  # that would be Bad (tm)
  FastCgiConfig -idle-timeout 20 -maxClassProcesses 1
  <Directory /var/local/fcgi>
    # Use the + so we don't clobber other options that
    # may be needed. You might want FollowSymLinks here
    Options +ExecCGI
  </Directory>
  AddType application/x-httpd-php5 .php
  AddHandler fastcgi-script .fcgi
  Action application/x-httpd-php5 /fcgi/php-cgi-wrapper.fcgi
</IfModule>

Now set up /var/local/fcgi/php-cgi-wrapper.fcgi:

mkdir -p /var/local/fcgi/
vi /var/local/fcgi/php-cgi-wrapper.fcgi

Paste in:

#!/bin/sh

# We like to use the same settings we formerly used for apache mod_php.
# You don't want this if your php.ini is in /etc
PHPRC="/etc/php5/apache2"
export PHPRC
# We can accommodate about 20 50mb processes on a 1GB slice. More than that
# will swap, making people wait and locking us out of our own box.
# Better idea: just make people wait to begin with
PHP_FCGI_CHILDREN=20
export PHP_FCGI_CHILDREN
exec /usr/bin/php-cgi

Note the figure 20, this can be adjusted for the box in question if it has more or less memory to offer. Also note that we use the same php.ini we used with mod_php (/etc/php5/apache2). You can change that if you think it appropriate but don’t forget to bring settings over.

Set permissions:

chmod -R 755 /var/local/fcgi

IMPORTANT: comment out any php_value lines in .htaccess files for the sites on this box, and move those directives to php.ini. Otherwise you will get errors form those sites and they won’t work.

If two sites on the same slice really must have separate php.ini settings, consider separate slices for the sites. It’s possible to set up more than one fastcgi pool, but that halves the resources available and so on.

# Commented out; moved to /etc/php/apache2/php.ini
# php_value arg_separator.output &

Disable mod_php5, enable mod_fastcgi in its place, enable mod_actions which is required by the above, and restart Apache:

a2dismod php5; a2enmod fastcgi; a2enmod actions; apache2ctl restart

Now test the site. Also check /var/log/error.log and/or error.logs for individual sites to make sure you see mentions of fastcgi and not mod_php.

If you get an error and you see this in the log:

[Thu Mar 04 02:51:33 2010] [alert] [client 69.141.215.7] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/local/fcgi/php-cgi-wrapper.fcgi": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive

There’s a chance it’s just a transitory problem; first-time fastcgi startups seem to be a bit temperamental, but we’ve had no problems in the long haul. Restart apache again:

apache2ctl restart

If it still doesn’t work reenable mod_php5:

a2dismod fastcgi; a2enmod php5; apache2ctl restart

And do some more reading.

Late-breaking news: PHP 5.3.3 and the new SAPI FPM

In addition to the built-in support for managing FastCGI child processes in php-cgi, PHP 5.3.3 introduces a new PHP binary, php-fpm, that offers a superior implementation of FastCGI support. You don’t have to switch if you are happy with the FastCGI support in php-cgi, which is fortunate because your operating system might not offer PHP 5.3.3 or its new php-fpm binary (aka the SAPI FPM) yet.

Setting up the new SAPI is not that different from setting up FastCGI with php-cgi, provided you can get past the probable need to compile it yourself (of course, by the time you read this, there may be a php-fpm package in your favorite Debian repository).

Interested parties should check out the new FastCGI Process Manager article on www.php.net.

Installing the Worker Thread MPM

Once you reach this point successfully, you are ready to switch Apache from “prefork” mode (which mod_php requires) to “worker thread” mode. Worker thread mode is blindingly, ridunculously fast at serving static stuff, and uses mod_fastcgi to take care of PHP. You can’t do this with mod_php enabled, so you must succeed in setting up fastcgi first (see above):

apt-get install apache2-mpm-worker

That will restart Apache by itself. But if not you can do so manually.

Now test the site again. If it doesn’t work, you can easily undo this:

apt-get install apache2-mpm-prefork

To verify success:

ps auxw | grep apache

Should show fewer processes (unless your operating system shows threads as processes). The processes you do have are running many threads (lightweight subprocesses that are exposed to each other’s memory etc. and generally less expensive).

ps auxw | grep php

Should show roughly 20 processes, plus a process to manage the others. Significantly more than 20 (say, 40+) indicates you goofed and didn’t set maxClassProcesses properly and you’re going to have a lot of PHP processes on your hands in a hurry, so fix that. (I’m assuming you chose 20 processes as your limit for PHP above.)

Now beat up on the site a bit. Remember to check any other sites on the same box, you just changed how all of your PHP sites on this box are run.

Now copy apc.php to the web folder of one of your sites and visit that URL to see a page giving stats about how well the cache is working. You should see a healthy percentage of cache hits if you have revisited some of the same pages or at least reused Symfony and Doctrine. If not check whether you ever enabled APC for this site. You can find apc.php in /usr/share/php/apc.php on some systems. Its absence doesn’t mean APC is not available.

You need to set up both mpm-worker and fastcgi to get the most benefit here. The worker MPM can serve zillions of static files in a hurry with minimal memory usage each. However, after setting up fastcgi, you can certainly crank up MaxClients and stick with the prefork MPM if you prefer.

Alternatives

APC alternatives: there are alternatives to the APC cache. These include eAccelerator and Zend Optimizer. I’m not an expert on the others, but APC will be standard in PHP 6.0, so I recommend standardizing on it now. For extreme cases (i.e. “we’re getting as big as Facebook and we have too much code to start thinking about improving the code itself in any serious way”), by all means consider HipHop. You can think of it as a distant relative of APC that provides a roughly 2x performance boost at a large cost in complexity and maintenance.

FastCGI alternatives: some will disagree with my recommendation of FastCGI. mod_cgid supposedly has better child process management… with the glaring exception that APC is not shared, ballooning your memory requirements in a seriously ugly way. Better alternatives include running the nginx (pronounced “engine X”) web server as a lightweight “front end” that serves static files directly and proxies other requests through to an Apache server running mod_php. One can also run Apache (without the fat mod_php module) as a front end for another instance of Apache. In these setups, the back-end Apache doesn’t accept connections except from the front-end one, and Apache’s reverse proxy features come into play. Personally I like the simplicity of Apache communicating directly with a master PHP process via the FastCGI protocol.

Conclusion

ERRRRRRRRRRRROOOOOMMMMMMM

Beep beep

ERRRRRRRRRRRRRRRRRROOMMMMMMMM

[HONK]

Zoom!

(*) Actually, one of Facebook’s stated goals for HipHop is to avoid rewriting and presumably hand-optimizing their existing PHP code. They have billyuns and billyuns of lines of PHP code, not all of it written by hardcore software jocks. So if a handful of smart folks can make mediocre PHP run 50% faster, that’s a big deal for Facebook. For most of us it makes more sense to pick the low-hanging fruit first (see all of the above), and then stop making separate SQL queries for every single darn object on the page, and then maybe think about HipHop. I’m not knocking HipHop here at all, it’s an impressive technical accomplishment. But if you’re starting with HipHop as your
first optimization, you’re not paying attention.

(**) Yes, you could use CSS sprites to speed these up, and given time you probably should, but in the short term let’s talk about ways to deliver them quickly without getting bogged down in hard-to-maintain optimizations. With HTTP keepalive working for us instead of against us, the performance difference between delivering 20 PNGs and one PNG is mostly on the browser side— still good to avoid, but not a brutal impact on your web server. One step at a time, okay?

(***) Yes, FastCGI is capable of running lots of things other than PHP, but this is a PHP article. So: neener.

(****) There’s nothing wrong with the FastCGI license. It’s a very boring MIT-style license that shouldn’t interfere with your commercial use.

We developed Apostrophe to scratch our own itch: we needed a CMS that our clients could use to add content to their sites in a variety of ways without inadvertently breaking the site design. So our CMS concentrates on the admin’s user experience, extending metaphors already present in the page. It’s about design even more than it’s about software architecture.

We also wanted to make sure it supported Symfony coding practices we were already having good success with. Therefore slots and engines implemented as Symfony modules, not new abstractions, and a clear lineage back to ideas from sfSimpleCMS.

We open sourced Apostrophe because we saw a win-win situation: other developers are in the same boat (their clients are also frustrated with Drupal administration, and they need a good solution today).

More eyes on our code, more bug reports, more suggestions equals a better system for everyone in the Apostrophe community.

Right now this is working for us in a big way, and it’s running in production on a number of client sites. Others on the apostrophenow google group are also happy with it (although they have valid criticisms that we are addressing), and they are building client sites that work. There’s no business case for us to screech to a stop and start doing something radically different.

Right now we have a team of seven people doing great work with Apostrophe and no interest in stopping that. Evolving it to be better, eventually rewriting it extensively in the Symfony 2.0 timeframe… sure! But unless the marketplace tells us otherwise, we’re going to keep doing good business, writing good code and sharing our work with those who like our approach.

Joomla, Wordpress and Drupal are all thought of as CMSes for PHP, but no one is telling them that they must merge. Why? People pick the project that works best for their needs. That’s as it should be. Software architecture isn’t the only differentiating factor between content management systems.

If you like Diem or Sympal’s approach better, heck, go ahead and use ‘em. A “there can be only one” approach leads to design by committee, and that is not always a good thing.