@agamemnon just checked in from Shiny Wooden Horse Disco Everybody Come Check It Out! via gowalla

@cassandra IT’S A TRAP!

@priam @cassandra SHUT UP ADMIRAL AKBAR THIS HORSE IS AWESOME

@agamemnon is now mayor of Sacked Ruin of Troy BOOYEAH! on #foursquare!

With frameworks like Symfony, you can build them faster still, and follow modern programming practices at the same time.

And Apostrophe strips away yet another layer of effort if your site calls for a content management system.

Yes, Java has more raw speed, all else being equal (which it never is). But as the LISP programmers used to say, “a moment of regret, a lifetime of convenience.”

Still, sooner or later success catches up with you and you want your site to cope with Serious Traffic… or cope with moderate traffic on a cheap virtual machine… or at the very least, not be dog-slow with just a handful of users on the system.

There’s a lot of advice out there about optimizing PHP code, some of it well worth your while. And there’s excitement about HipHop, Facebook’s new native code compiler for PHP. But these are drastic steps that require you to rewrite your code or adopt less proven and more awkward ways of delivering your code.

Justified? Sure, sometimes, on the biggest projects in the world (like Facebook) (*). But as Donald Knuth says, “premature optimization is the root of all evil.” That’s because tweaking your code for speed’s sake usually makes it harder to maintain and less adaptable to new requirements.

What most developers don’t realize is that there are three major factors that typically slow down PHP projects based on frameworks (like Symfony or, sigh, Drupal) so much that code profiling and database query redesign don’t even have a chance to become relevant factors. Fix these things first before you worry about other issues:

1. Compiling code over and over and over. Would you wait for your Mac to recompile MacOS X from source code every time you boot it up? Of course not. How about every time you fill out a dialog box? That’s pretty much what you’re doing every time you access a PHP-driven website that doesn’t use a bytecode cache.

2. Waiting and waiting and waiting for web browsers to make another request, pinning down web server processes that your other users need. By default Apache usually lets browsers hold on to a connection for up to 15 seconds just in case they ask for more. This is a good thing in many ways, but 15 seconds is far too long. Which leads us to #3:

3. Tying up a “fat” web server process with PHP on board for every request, even requests for the zillions of little static PNGs that probably make up your page design. (**) A typical Apache web server configuration with mod_php suffers from this flaw, fatally limiting the number of simultaneous users you can handle.

So what can we do about these problems? Quite a bit as it turns out. I’ll start with the low-hanging fruit and move on to the tougher stuff. The fascinating common thread with all of these suggestions: no changes at all to your PHP code.

Stop Recompiling Your World: APC

APC, aka the Alternative PHP Cache, is a bytecode cache. What does that mean and what good does it do?

Today, when users visit your website, your web server struggles to translate all of your PHP code into a more manageable “bytecode.” And only then does it actually run the bytecode and output the page.

Bytecode isn’t true native machine code and doesn’t run as fast as compiled Java, and certainly not as fast as compiled C. That’s why Facebook’s HipHop project seeks to convert PHP code to C++, and then compile that to native code. But that’s a drastic step with far-reaching consequences for your project. Before you think about HipHop, try a simpler fix: stop throwing away the bytecode and starting over from zero on every new request that comes to the site!

In our experience, enabling APC reduces both memory usage and the time required to respond to a request by vast amounts. Even a simple action in Symfony or Drupal can implicitly require that quite a bit of PHP code be compiled. With APC, all of that code stays compiled, and that simple action actually runs quickly and simply.

APC is simply indispensable for serious PHP programming. Turn it on. Make sure it’s really working. And enjoy dramatic improvements in your social life, hair and skin.

Installing APC

Depending on your system, APC may already be installed or even, if you are very lucky, activated correctly. Virtual machine hosting from ServerGrove comes with APC “out of the box.” Most shared hosting probably won’t because it increases the amount of memory pinned down for each user on the shared server, but if you care about your website’s speed, you’re not using shared hosting. Get a small, cheap virtual machine from ServerGrove instead (or if you must, do it the hard way and set up a Slicehost or linode virtual machine; you’ll learn a lot about system administration).

On our slicehost VMs, we found it necessary to install APC with pecl, the PHP package manager for extensions (like PEAR, but different in that they are not written in plain PHP):

pecl install apc

We then enable it in php.ini (you won’t be able to do this in a .htaccess file):

; Must enable APC otherwise we compile all of Symfony on every hit
extension=apc.so
; Give APC enough shared memory to handle Symfony, etc.
apc.shm_size = 48
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

Then restart Apache:

apachectl restart

And you’re off to the races. The first page access will be as slow as ever, but subsequent accesses will be much, much faster. You can also check whether you’re saving memory and time using Symfony’s debugging toolbar.

Your system also likely has an APC status page that you can copy to your website to check out the current status of APC. It’s fun to play with.

Dialing Back Keepalive: Too Much of a Good Thing

In the beginning there was NCSA Mosaic, and it was pretty cool, all things considered. But it wasn’t fast.

Mosaic made only one request of a web server at a time. Trying to do more probably seemed rude. Besides, lots of web servers couldn’t cope with more than one request a time at all. But that would only happen if two users came along at once, and that was never gonna happen, right?

Ah, those were the (sad, lame, pathetic) days. Scroll to the present. Typical web browsers first request the page, say “hang on a minute,” open perhaps five more connections besides and then make note of all the images, JavaScript files, CSS files and other embedded bric-a-brac and start requesting all of those things. Consecutively and in parallel. Really a lot.

The Apache web server is actually very good at handling this. But by default, it is very generous: it lets the browser leave it hangin’ for fifteen seconds. That means an Apache process and all of the memory it requires is tied up for all of that time, just doin’ nothin’. And that’s bad news.

Fortunately it’s very easy to fix this in your httpd.conf file (actually, in our setups, this is usually in apache2.conf):

KeepAliveTimeout 2

Restart Apache:

apachectl restart

And you’ll find that your site can suddenly cope with higher loads. That’s because your’re not tying up valuable resources waiting around for more requests from browsers that are already using and reusing five other connections to your server. The browser is welcome to promptly request more files and finish its business with you, but not to stand around twiddling its thumbs while you wonder if you have a date or not.

You can turn keepalive off entirely, but doing so forces the browser to “call back” for each and every little image file on your page, which is a terrible idea. So use it… just don’t let it abuse you.

MaxClients: Does This mod_php Make My Butt Look Big?

mod_php is fast. Unlike the old-fashioned “PHP as CGI” approach, mod_php keeps the PHP interpreter itself loaded in memory and ready to rock at all times, and that’s good.

But mod_php is also a pig. For a big-deal web application that wrangles lots of database objects, or makes thumbnails of uploaded images, or does pretty much anything cool, it’s not uncommon for a PHP process to use 50MB of memory or more. More miserly code can still hit 30MB without much difficulty.

We first ran into this problem when we wanted to move smaller client sites onto small VMs, so that they could have well-secured, highly-reliable web hosting at an affordable price. They tended to bog down and become unusable quickly.

We discovered that dialing back keepalive helped tremendously with the problem. And we also learned to do the math and figure out how many reasonably fat PHP Apache processes would really fit in the memory available to the slice, and configure Apache to run no more than that. In httpd.conf:

MaxClients          20

This may seem drastic (OMG! I can’t serve more than 20 simultaneous users!) but here’s the thing: you already can’t. If you try, if you leave this set to its much higher default, the result will be a VM crushed by the need to swap out all of these Apache processes to virtual memory while they are still trying to do work. As the operating systems geeks say, the “resident set size” of your Apache and PHP processes is bigger than your actual memory. And when that happens you’re SOL.

When you set MaxClients to a realistic number, what happens instead is this:

You can handle up to 20 connections at once gracefully.
That 21st connection queues up and is dealt with soon after.
If the average load doesn’t exceed 20 connections, you’re fine.

If too many connections queue up, some people will not get through. But you will still be able to get through to other services on the VM… like the ssh connection you use to administer it and change these settings. Without that, you’re stuck in a seesaw battle, shutting down your website, tweaking settings, firing it up again and watching it gradually become overwhelmed.

What Is The Right MaxClients For Me?

Take the amount of RAM in your server (for instance, a 256MB slicehost VM). Subtract a bit of overhead (at least 50MB, ideally more, bigger VMs are better that way). Now divide by the largest PHP process you typically see when running “top” or “ps auxw | grep php”. For instance, if your processes top off around 30MB, you can set MaxClients to (256-30) divided by 30. Round down and you have a MaxClients setting of 7.

One snag: some operating systems show shared memory (like the memory used by APC) as memory in use by every one of the Apache processes, even though it is really only allocated once and shared by all of them. If your APC cache is set to 40MB and all of your PHP processes seem 40MB overweight, you can increase MaxClients somewhat to allow for this.

Yes, you can use swap space (virtual memory) to go higher, but this just forces the machine to swap, using disk space in place of memory to do hard things it needs to do quickly and freezing you out of your own machine. So don’t do that.

PHP, I Love You, But Could You Sleep Downstairs?

Dialing back the number of Apache processes is a helpful step that allows your server to say no with grace and style when there is too much traffic. But the underlying problem remains: we can’t deal with a lot of requests at once. And the reason we can’t do that is PHP. So we’re stuck. Or are we?

Our Apache processes are “fat” in terms of memory use because of the memory taken up by PHP. But most of the requests we’re handling are for little PNG files and not-so-little JPEG files and CSS files and JavaScript files… all of which Apache can serve up handily with about three bytes of memory and a piece of string.

The obvious solution is to serve PHP requests and static files separately. And many sites do this explicitly, using absolute URLs pointing at a different subdomain for static files. For example:

<img src=”http://static.example.com/foo.png” />

Now, this is a fine thing and I have nothing against it. When you design things this way you can use entirely separate machines to serve these requests if you wish, and that’s pretty exciting from a performance perspective. Many bigger sites find this well worth their while.

But you are also complicating things. And you are forced to change working code, which introduces the potential for new and exciting bugs.

There’s another approach (well, one of many): FastCGI. FastCGI is an enhanced version of the CGI protocol that appeared quite early in the evolution of the web but caught on only with those most concerned about performance. I must admit I am a very recent convert, ironic since FastCGI is a bit out of vogue today. But there are solid reasons to consider using it.

FastCGI has three big advantages:

1. FastCGI runs PHP processes separately from Apache. That means Apache can stay lean and mean, not only running much smaller processes (and therefore perhaps many more of them) but even running in a separate “worker thread” that outperforms the separate-processes mode by a long mile. This is a big, big win.

2. It can reuse a single process for new PHP requests (***), avoiding the need to restart the PHP interpreter. That’s nice, but mod_php can do the same trick, and mod_php is capable of sharing APC cache memory between all of its processes. And FastCGI can’t do that… or it wouldn’t be able to without a little help from PHP:

3. PHP comes with a special FastCGI mode in which it manages several child processes of its own, and allows all of those processes to share a single APC cache.

Put these advantages together and you have the holy grail of PHP hosting: PHP running separately from Apache, but still maintaining all of the performance advantages of the APC cache. And we can do it without changing a line of PHP.

Installing FastCGI

There’s one catch: Apache 2.0 ships with mod_cgid, a rewritten version of FastCGI that complies with the official Apache license, instead of the oh-so-fractionally-different FastCGI license (****). And it’s nice and all, except that it completely lacks the third advantage of FastCGI I mentioned above. If you want the benefits I’m describing here, you have to use the real mod_fastcgi. Accept no substitutes.

These directions work for slicehost Ubuntu slices and other Ubuntu-based Linux. They will not work exactly as-is for servergrove which is centos-based:

Edit /etc/apt/sources.list
Add multiverse, which includes software with insufficiently “open” licenses, such as mod_fastcgi. Change intrepid to the ubuntu version on your box, it’ll be mentioned all over sources.list:

# tom@punkave.com: we want multiverse support so we can have real mod_fastcgi
deb http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid multiverse
deb http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ intrepid-updates multiverse

Now do:

apt-get update
apt-get install libapache2-mod-fastcgi

Now edit /etc/apache2/mods-available/fastcgi.conf to read:

<IfModule mod_fastcgi.c>
  # One shared PHP-managed fastcgi for all sites
  Alias /fcgi /var/local/fcgi
  # IMPORTANT: without this we get more than one instance
  # of our wrapper, which itself spawns 20 PHP processes, so
  # that would be Bad (tm)
  FastCgiConfig -idle-timeout 20 -maxClassProcesses 1
  <Directory /var/local/fcgi>
    # Use the + so we don't clobber other options that
    # may be needed. You might want FollowSymLinks here
    Options +ExecCGI
  </Directory>
  AddType application/x-httpd-php5 .php
  AddHandler fastcgi-script .fcgi
  Action application/x-httpd-php5 /fcgi/php-cgi-wrapper.fcgi
</IfModule>

Now set up /var/local/fcgi/php-cgi-wrapper.fcgi:

mkdir -p /var/local/fcgi/
vi /var/local/fcgi/php-cgi-wrapper.fcgi

Paste in:

#!/bin/sh

# We like to use the same settings we formerly used for apache mod_php.
# You don't want this if your php.ini is in /etc
PHPRC="/etc/php5/apache2"
export PHPRC
# We can accommodate about 20 50mb processes on a 1GB slice. More than that
# will swap, making people wait and locking us out of our own box.
# Better idea: just make people wait to begin with
PHP_FCGI_CHILDREN=20
export PHP_FCGI_CHILDREN
exec /usr/bin/php-cgi

Note the figure 20, this can be adjusted for the box in question if it has more or less memory to offer. Also note that we use the same php.ini we used with mod_php (/etc/php5/apache2). You can change that if you think it appropriate but don’t forget to bring settings over.

Set permissions:

chmod -R 755 /var/local/fcgi

IMPORTANT: comment out any php_value lines in .htaccess files for the sites on this box, and move those directives to php.ini. Otherwise you will get errors form those sites and they won’t work.

If two sites on the same slice really must have separate php.ini settings, consider separate slices for the sites. It’s possible to set up more than one fastcgi pool, but that halves the resources available and so on.

# Commented out; moved to /etc/php/apache2/php.ini
# php_value arg_separator.output &

Disable mod_php5, enable mod_fastcgi in its place, enable mod_actions which is required by the above, and restart Apache:

a2dismod php5; a2enmod fastcgi; a2enmod actions; apache2ctl restart

Now test the site. Also check /var/log/error.log and/or error.logs for individual sites to make sure you see mentions of fastcgi and not mod_php.

If you get an error and you see this in the log:

[Thu Mar 04 02:51:33 2010] [alert] [client 69.141.215.7] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/local/fcgi/php-cgi-wrapper.fcgi": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive

There’s a chance it’s just a transitory problem; first-time fastcgi startups seem to be a bit temperamental, but we’ve had no problems in the long haul. Restart apache again:

apache2ctl restart

If it still doesn’t work reenable mod_php5:

a2dismod fastcgi; a2enmod php5; apache2ctl restart

And do some more reading.

Installing the Worker Thread MPM

Once you reach this point successfully, you are ready to switch Apache from “prefork” mode (which mod_php requires) to “worker thread” mode. Worker thread mode is blindingly, ridunculously fast at serving static stuff, and uses mod_fastcgi to take care of PHP. You can’t do this with mod_php enabled, so you must succeed in setting up fastcgi first (see above):

apt-get install apache2-mpm-worker

That will restart Apache by itself. But if not you can do so manually.

Now test the site again. If it doesn’t work, you can easily undo this:

apt-get install apache2-mpm-prefork

To verify success:

ps auxw | grep apache

Should show fewer processes (unless your operating system shows threads as processes). The processes you do have are running many threads (lightweight subprocesses that are exposed to each other’s memory etc. and generally less expensive).

ps auxw | grep php

Should show roughly 20 processes, plus a process to manage the others. Significantly more than 20 (say, 40+) indicates you goofed and didn’t set maxClassProcesses properly and you’re going to have a lot of PHP processes on your hands in a hurry, so fix that. (I’m assuming you chose 20 processes as your limit for PHP above.)

Now beat up on the site a bit. Remember to check any other sites on the same box, you just changed how all of your PHP sites on this box are run.

Now copy apc.php to the web folder of one of your sites and visit that URL to see a page giving stats about how well the cache is working. You should see a healthy percentage of cache hits if you have revisited some of the same pages or at least reused Symfony and Doctrine. If not check whether you ever enabled APC for this site. You can find apc.php in /usr/share/php/apc.php on some systems. Its absence doesn’t mean APC is not available.

You need to set up both mpm-worker and fastcgi to get the most benefit here. The worker MPM can serve zillions of static files in a hurry with minimal memory usage each. However, after setting up fastcgi, you can certainly crank up MaxClients and stick with the prefork MPM if you prefer.

Alternatives

APC alternatives: there are alternatives to the APC cache. These include eAccelerator and Zend Optimizer. I’m not an expert on the others, but APC will be standard in PHP 6.0, so I recommend standardizing on it now. For extreme cases (i.e. “we’re getting as big as Facebook and we have too much code to start thinking about improving the code itself in any serious way”), by all means consider HipHop. You can think of it as a distant relative of APC that provides a roughly 2x performance boost at a large cost in complexity and maintenance.

FastCGI alternatives: some will disagree with my recommendation of FastCGI. mod_cgid supposedly has better child process management… with the glaring exception that APC is not shared, ballooning your memory requirements in a seriously ugly way. Better alternatives include running the nginx (pronounced “engine X”) web server as a lightweight “front end” that serves static files directly and proxies other requests through to an Apache server running mod_php. One can also run Apache (without the fat mod_php module) as a front end for another instance of Apache. In these setups, the back-end Apache doesn’t accept connections except from the front-end one, and Apache’s reverse proxy features come into play. Personally I like the simplicity of Apache communicating directly with a master PHP process via the FastCGI protocol.

Conclusion

ERRRRRRRRRRRROOOOOMMMMMMM

Beep beep

ERRRRRRRRRRRRRRRRRROOMMMMMMMM

[HONK]

Zoom!

(*) Actually, one of Facebook’s stated goals for HipHop is to avoid rewriting and presumably hand-optimizing their existing PHP code. They have billyuns and billyuns of lines of PHP code, not all of it written by hardcore software jocks. So if a handful of smart folks can make mediocre PHP run 50% faster, that’s a big deal for Facebook. For most of us it makes more sense to pick the low-hanging fruit first (see all of the above), and then top making separate SQL queries for every single darn object on the page, and then maybe think about HipHop. I’m not knocking HipHop here at all, it’s an impressive technical accomplishment. But if you’re starting with HipHop as your
first optimization, you’re not paying attention.

(**) Yes, you could use CSS sprites to speed these up, and given time you probably should, but in the short term let’s talk about ways to deliver them quickly without getting bogged down in hard-to-maintain optimizations. With HTTP keepalive working for us instead of against us, the performance difference between delivering 20 PNGs and one PNG is mostly on the browser side— still good to avoid, but not a brutal impact on your web server. One step at a time, okay?

(***) Yes, FastCGI is capable of running lots of things other than PHP, but this is a PHP article. So: neener.

(****) There’s nothing wrong with the FastCGI license. It’s a very boring MIT-style license that shouldn’t interfere with your commercial use.

We developed Apostrophe to scratch our own itch: we needed a CMS that our clients could use to add content to their sites in a variety of ways without inadvertently breaking the site design. So our CMS concentrates on the admin’s user experience, extending metaphors already present in the page. It’s about design even more than it’s about software architecture.

We also wanted to make sure it supported Symfony coding practices we were already having good success with. Therefore slots and engines implemented as Symfony modules, not new abstractions, and a clear lineage back to ideas from sfSimpleCMS.

We open sourced Apostrophe because we saw a win-win situation: other developers are in the same boat (their clients are also frustrated with Drupal administration, and they need a good solution today).

More eyes on our code, more bug reports, more suggestions equals a better system for everyone in the Apostrophe community.

Right now this is working for us in a big way, and it’s running in production on a number of client sites. Others on the apostrophenow google group are also happy with it (although they have valid criticisms that we are addressing), and they are building client sites that work. There’s no business case for us to screech to a stop and start doing something radically different.

Right now we have a team of seven people doing great work with Apostrophe and no interest in stopping that. Evolving it to be better, eventually rewriting it extensively in the Symfony 2.0 timeframe… sure! But unless the marketplace tells us otherwise, we’re going to keep doing good business, writing good code and sharing our work with those who like our approach.

Joomla, Wordpress and Drupal are all thought of as CMSes for PHP, but no one is telling them that they must merge. Why? People pick the project that works best for their needs. That’s as it should be. Software architecture isn’t the only differentiating factor between content management systems.

If you like Diem or Sympal’s approach better, heck, go ahead and use ‘em. A “there can be only one” approach leads to design by committee, and that is not always a good thing.

My own opinions are in parentheses.

They are setting up now.

“Symfony 2.0 Reloaded” Preview Release

“I’ve decided to do the session in French” heh

In 10 minutes you will discover the shiny new version of Symfony, Symfony 2.0. Symfony 2 is just awesome. But first a bit of history.

Symfony 1.0 released in January 2007. Started as glue between existing libraries: Mojavi, Propel, Pardo i18n, …

“How many of you are still using 1.0?” [Some hands go up] “Ohhhhh!” [Hey, I raised my hand too, I have to support existing projects after all, heh. Our new stuff is all 1.4]

With Symfony 1.2 in November 2008, decoupled but cohesive components wre rolled out: forms, routing cache, YAML, ORMs… but still the same core.

Symfony 1.4 – November 2009. Added polish on existing features. Removed the support for deprecated features. Current LTS released, maintained until late 2012.

What is Symfony 2.0? The next version of course… except Symfony now takes an ‘S’ instead of an ’s’. That’s the big difference!

Talk about Symfony 2 with a capital S please! Symfony 1 (uppercase) does not make any sense, symfony 2 does not make sense! It’s all about Symfony 2. “I’m quite picky.”

“Same philosophy, just better.” We’ve learned a lot from our mistakes. We’ve learned a lot… we have problems with Symfony 1, but not all are fixable. It requires breaking backwards compatibility. So we need to go forward.

Symfony 2 is still an MVC framework.

It’s probably more of a Fabien-style framework than anything else (his words not mine (: ).

MTV: Model, Template and View?

Highly configurable and extensible. Same Symfony Components (DI, events, request, etc), same great developer tools, full featured.

Everything is upgraded to PHP 5.3. (Great, I hope they debug PHP 5.3 by the time Symfony 2.0 is complete!)

OK, but why a major version # change then?

Because it has a brand new low-level architecture to fix some of the deeper issues with Symfony 1.x.

New best practices and concepts, built with speed and flexibility in mind. Based on dependency injection container, a new plugin system.

Namespaces and anonymous functions will be used.

A Quick Tour

namespace Application\HelloBundle\Controller;
use Symfony\Framework\WeBundle\Controller;

class HelloControlelr extends Controller
{
  public function indexAction($name)
  {
    return $this->render('HelloBundle:Hello:index', array('name' => $name));
  }
}

&lt!-- No magic for layouts this is an explicit layout extension in the template -->
<?php $view->extend('HelloBundle::layout') ?>

Hello <?php echo $name ?>!

Layout:

<html>
<head>....
<body>
<?php $view->slots->output('_content') ?>
</body>
</html>

Helpers are objects

hello:
  pattern: /hello/:name
  defaults:
    _bundle: HelloBundle
    _controller: Hello
   _action: index

:name becomes the first argument to the action. If you have more in the route, they become additional arguments to the action. No $request parameter.

Extremely Configurable

Why? Because it is all managed by a Dependency Injection Container. “I won’t talk about the DI container today” (aw, I wish someone had explained it this week; it was taken as a given by an earlier presentation today; I’ve read about it but many haven’t had a chance yet I’m sure).

The DI Container replaces “a lot of symfony 1 things:”

sfConfig
All config handlers
sfProjectConfiguration
sfContext (no singleton anymore!)
The configuration cache system (!!)
And some more

In one easy-to-master, unified, and cohesive package.

databases.yml can be renamed. You can create just one big config file if you want, or have several small ones.

Name your configuration files any way you want. Use PHP, XML, YAML or even INI format.

You can do raw PHP rather well:

$configuration = new BuilderConfiguration();
$configuration->addResource(new FileResource(__FILE__));

$configuration->mergeExtension('web.user', array('default_culture' => 'fr', 'session' => array('name => 'SYMFONY')...); etc

Same thing shown in YAML and XML. XML syntax is rather consistent, not much more verbose than YAML anymore.

$configuration->mergeExtension('swift.mailer', array('transport' => 'gmail', 'username' -=> 'fabien.potencier', 'password' => 'xxxxxx', ... ));

XMLNS schema definitions for lots of things

Autocompletion based on schemas demonstrated in NetBeans IDE

Inherit configuration as much as you want

You can have a master configuration and then configurations for each environment for instance. Symfony 1 had several defined levels. Framework, project, application, module.

In Symfony 2 it’s up to you. You can have as many as you want, and mix and match config files written in different formats, which is useful when using third-party plugins.

<imports>
<import resource="parent.xml" />
<import resource="config.yml" />
<import resource="parameters.ini" />
</imports>

<zend:logger
priority="debug"
/>

<web:debug
toolbar="%kernel.debug"
ide="textmate"
/>

Which format to use? Internally they only use XML because it has autocomplete, validation, help etc. etc. due to schemas. Fabien endorses XML.

Store Sensitive Settings Outside Of Your Project (database passwords… clever rsync_exclude tricks go away?)

Old way (in 2 syntax):

<doctrine:dbal
dbname="sfweb"
username="root"
password="HeyThisIsTotallyPublic"
/>

New way:

password=%SYMFONY_DOCTRINE_DBAL_PASSWORD%

And set:

SetEnv SYMFONY_DOCTRINE_DBAL_PASSWORD “foobar”

(is that literally a shell thing?)

Creating DIC Extensions Is Insanely Simple

Very fast thanks to a smart caching mechanism – always knows when to flush the cache (!)

Whenever you do something that requires creating the cache, Symfony knows it must flush and recreate the cache.

The code in the cache is PHPDoc, seriously readable code

The code to fetch something from the DIC is as fast as the code you’d write yourself (he showed us; there are no unnecessary layers of cruft)

The DIC can manage any PHP object: POPO (?)

“Plugins…”

Good in 1.x, better in 2.0

“So much better we need a new name.” They are now called Bundles.

Why? Because they are first class citizens in Symfony 2.0. In Symfony 1, they added the plugin system late in the game. They had applications and plugins and there was a question as to what should be what. In 2.0 everything is a bundle. Core features, third-party code, application code. The framework itself is made up of bundles. Doctrine is a bundle, etc.

Default directory structure:

app/
src/
web/

You don’t have to adhere to it.

app/
  AppKernel.php
  cache/
  config/
  console
  logs/

src/
  autoload.php
  Application/
  Bundle/
  vendor/
    doctrine/
    swiftmailer/
    symfony/
    zend/

Note that Zend is on the shortlist, but in “vendor” (and so is symfony).

web/
  index.php
  index_dev.php

Controllers.

A bundle dir looks like this:

.../
  SomeBundle/
  Bundle.php
  Controller/
  Model/
  Resources/
    config/
    views/

Templates, data, views, configuration, js, etc. live in Resources.

public function registerBundleDirs()
{
  return array(
    'Application' => __DIR__ . '/../src/Application',
    same for bundle
    similar for Symfony Framework pointing to its bundle dir, giving it a prettier name on the left, using namespaces
}

$this->render('SomeBundle:Hello:index', $params)

SomeBundle is the bundle’s declared name, not something based on position in the filesystem. Looks for a matching bundle (much as it now looks for modules in plugins and the app).

Could be in:

Applciation\SomeBundle
Bundle\SomeBundle
Symfony\Framework\SomeBundle

Extend things by making one that appears earlier on that list.

In Symfony 1 to extend a plugin you must muck about with a BaseaActions class to make aActions extensible. In Symfony 2 it is natural. Just create another bundle. (? How does a bundle extend a bundle exactly? Is it about the order you register them in? Are you extending classes or is this a DI trick?)

“Less concepts… but more powerful ones”

Symfony 1 View Layer

A layout is just another tempalte with __content as a special slot

A partial is just at emplate you embed in another one

A component is just another action

<?php $view->output('BlogBundle:Post:list', array('posts' => $posts)) ?>

<?php $view->actions->output('BlogBundle:Post:list', array('limit' => 2)) ?>

So back to the layout… _content is just a slot now, not something special.

Big and small improvements.

Multiple levels of layouts. You can extend the template, but you can also extend the layout as much as you want… partials can be decorated too. Better logging.

INFO: Matched route “blog_home” (parameters: array(‘_bundle’ => ‘BlogBundle’ ….))

Errors are called out better with an ERR prefix (there were supposed to be multiple levels in 1.x right?)

<zend:logger priority="debug" />

Great exception error pages

They integrate with your editor if you have the symfony firebug bar, very nice

“Content of the Output” available in the event of an error! Big improvement over 1.x. You can see the var_dumps before the error etc.

“An Even Better Web Debug Toolbar” (oops it was “Event Better” I was very puzzled (: )

The web debug toolbar is at the bottom of the page. You can look at it when you want to, otherwise out of your way.

“Everythign you need is at the bottom of the screen.”

symfony 2.0.0-DEV php 5.3.1/xdebug/accel blog/dev/debug default/html/200/text/html 6ms 1280kb [db] 0

“default” is the route that matched.

2.x is web designer “friendly.” No more apps/frontend/modules/foobar/templates before you can find what you are working on. You can have one templates folder.

app/
  views/
    BlogBundle/
    Post/
      index.php
 And then other bundles with their subfolders

(That looks familiar actually…? But he’s saying there is one templates dir…? Ah because a bundle is equivalent to a plugin not just a module. All templates for a plugin-scale thing are now in one folder)

Symfony 2 Is A Lazy Framework

use Symfony\Foundation\UniveralClassLoader;

$loader = new UniversalClassLoader();
$loader->registerNamespaces(Symfony, Application, Bundle, Doctrine with their root paths in a hash));
$loader->registerPrefixes(array('Swift_' ... ));

Lazy-loading of services

In 1.x factories are created by default. No choice. In 2.x, if you don’t use a user object, you won’t get one. If you don’t need request, you don’t get it.

Lazy-loading of listeners

A bunch of events, a lot of different listeners. In Symfony 2 we only create listeners if an event is notified. You can register as many listeners as you want, the overhead is small.

Lazy-loading of helpers

<?php echo $view->router->generate('blog_post', array('id' => $post->getId())) ?>

The router helper doesn’t get created unless I use it.

(Note that $view is a new variable present in view layer code like partials and templates. Helpers are objects, and view itself is an object. No more pretending you don’t need -> in templates (we all used it anyway with the model).)

Symfony 2 routing system has similar features but the implementation is very different. Compiles down to plain PHP code.

You can use Apache for Routing matching for extreme speed (!).

A Very Fast Development Environment

Again, no more clear cache, it’s automatic in dev environment. A metafile references the files that are relevant to the cache, and when they get touched Symfony can check very quickly whether it must clear the cache.

You can also stash routes in a db, and Symfony also knows if it needs to rebuild based on that.

And that’s Symfony 2, “we have barely scratched the face” of Symfony 2.0.

Easy to learn
Easy to use
Extensible at will

“People will tell you that Symfony 1 is slow, hard to learn, complex. I think they are wrong. But I think Symfony 2 is easy to learn, easy to use, and extensible at will. I want to lower the barrier to entry of Symfony 2.0. I want it to be usable for very small websites and very big ones. I want people to be able to grasp Symfony 2.0 in a matter of minutes. Probably 20 minutes. I want people to be able to understand and be fluent in a matter of hours. Perhaps a day, not more. Everything is more explicit. Less what-the-fuck effects” (borrowing Jon Wage’s line from yesterday heh). “You can change everything in Symfony 2.0. If something is not tasty enough for you, you can swap the component and use something else. You can change the whole directory structure if you want, create a whole application in one file if you want. You should be able to guess new concepts without reading the documentation, guess the thing you don’t know. That’s possible because we have fewer concepts. Done with two or three concepts, easier to guess how it works and how it would work for your specific case.

Of course we have all of the great features of Symfony 1…

But Symfony 2 should be slow right? No. It is

Fast as hell

This is the fastest framework ever created. It can be seven times faster than any other framework for a simple hello world application but I don’t care. The last time a customer asked for a hello world application was in the early nineties so that benchmark does not make sense. So I created a benchmark with a more typical application using all the layers of a typical framework. Models, views, controller, routing. I did benchmarks on a simple application. It’s a hello ‘name’ application, so there is a route, /hello/someone. There is a layout with 15 links and a menu with 15 routes… there are partials.

So we use templates and partials (setTemplate()), links for the routing, layout, but no session or database in this case. We are trying to measure the speed of the framework itself.

If you do this, Symfony 2.0 is 2x faster than Solar 1.0.0, the fastest framework for simple applications. So that’s great.

Symfony 2.0 is 2x faster than Symfony 1.2, 2.5x faster than Zend, 4x faster than Lithium, 6x faster than CakePHP 1.2.6. 60x faster than Flow3 (and by the way notice that this means Symfony 1.x is pretty fast relative to most of these).

Symfony 2.0 uses half the memory needed by both Symfony 1 and Zend Framework. Scales better than any other framework.

Stuff still missing:

Controller’s nice default pages. Cache via ZF – DI extension coming soon. Form/Validation/Widget: can use 1.4 version for now. Admin generator, ditto. Doctrine 2.0 needs its upper layers but the DBAL layer is already impressive. There is no handy bundle installer yet.

Not for production projects yet, quite stable but he needs to be able to change his mind and make big changes for now. Final release date 2010.

For bleeding edge news (ha!) follow @fabpot, and also on Github, github.com/fabpot.

He’s going to release the code (pull a copy?) live.

He’s on a Mac too small for him to access the usual VPN icon so he’s asked the audience for help finding it through system preferences hee! Good he found it. We’ll see if the network works for him…

He’s going through the ugly validation of the wifi here… “what-a-fuck?”

“Okay so you won’t have the code today… Sorry about that. No, I’ll do my best to push the code in about an hour. I will also deploy symfony-reloaded.org, a temporary website about Symfony 2.0.

I can tell you that at the end of the tutorial, you will be able to create great applications with Symfony 2.0. I am a bit frustrated now because I have so many things to tell you. I really think Symfony 2.0 is awesome. So if you have any good questions…”

Standing Ovation

Q. “Are you trying to reinvent Zend Framework 1.0?” Heh. This is a reference to a dumb question asked of Jon Wage yesterday. If I heard right.

Q. “How can we balance future development on 1.x and the new 2.0? What should we do [now] as community working on Symfony framework?”

A. If you start a new project today use 1.4. If you respect best practices and separate model view and controllers and controllers are thin and model is fat and well-coupled templates it will be easier to migrate to 2.0. Don’t try to use 2.0 for a site deploying before end of the year. It’s too early. But Symfony 2 is so flexible. We will find ways to ease the upgrade path. You can also move to PHP 5.3 today.

Q. About bundle loading: one of the targets is less magic but you first look at the application bundle and things like that. Why keep that sort of magic?

A. It’s not magic at all.

Q. It’s not exclusive.

A. It is. You need… a small amount of magic. Just a small amount. But you can change the directories, the name of the namespaces, you can have just one namespace if you want.

Q. Can you have sub-bundles without the project level having to know about them explicitly? Can you put a bundle into another bundle?

A. Doesn’t make sense. Too complex. You can… yeah why not? Not a big deal… doesn’t feel right.

Q. (Me): If I write a bundle, and someone wants to extend an action in my bundle, how do they do that?

A. They just extend your class, and make sure their bundle loads first. (I think I get it: now that we get to declare functional names for our stuff, independent of class names, there’s no barrier to extending someone else’s class and using the same functional name so it replaces it. As for loading order, you can have more than one bundles folder and call one of them ‘overrides’ and put it first, or similar.)

Q. Will there be documentation or support soon to help us create bundles that will work for both 1.4 and 2.0?

A. No. Controllers are too different. No more actions.class.php. Very different, with namespaces. You can create a plugin that is also a bundle but there’d be a lot of duplicate code.

Q. But what about the impact on migration of existing code?

A. It’s too early to say but there will be great news to announce re: new bundles. One of the problems with Symfony 1 plugins is the quality. We have a lot of plugins for 1.x but a lot of them are just not really usable, to say the least. This will be better. I can’t say more right now.

Q. Do we have to enable bundles? Is it possible to set up dependencies between different bundles?

[Fabien opens his sandbox in textmate]

[Points out the registerBundles method seen above]

You need to register bundles.

class Hellokernel extends Kernel

Symfony is now an operating system! Okay no.

Fabien: “if you want, if you want, if you really want…” somebody remix this please (:

… And that’s it! Great stuff. Symfony 2.0 will be a big improvement. My next step is to wrestle DI all the way to the ground (:

“Not the same old PHP ORM”

100% re-written codebase for PHP 5.3

“Are you scared? You shouldn’t be! Change is a good thing!”

5,000 records hydrate in 1.4 seconds. Whee!

10,000 records in 3.5 seconds.

Twice the data and still faster than Doctrine 1.

% improvement over Doctrine 2 is big. PHP 5.3 is a big win with a heavily OO framework. Better optimized hydration algorithm. New query and result caching implementations. All around more explicit and less magical code. Killed the magical aspect of Doctrine 1.

“The only valid measurement of code quality: WTFs/minute”

Good code: “WTF….. WTF”

Bad code: “WTF WTF WTF WTF WTF WTF”

Doctrine 1 magical features are both a blessing and a curse

Magic is great when it works but also causes pain. When it doesn’t work it is hard to debug. Edge cases are hard to fix. Edge cases are hard to work around. Everything is okay until you try to go outside the magic box. Magic is slow.

Replacement: OOP. Object composition, inheritance, aggregation, containment, encapsulation, etc.

Will it have behaviors? Yes and no. No “model behaviors.” Made up to work with Doctrine’s “intrusive architecture.” Tries to do things PHP doesn’t allow, results in a lot of problems (like sfMixin did).

If you could do it in 1 you can do it in 2, just differnetly. Natural OOP that wraps/extends Doctrine code or is meant to be wrapped or extended by your entities. (Injection? Factories? How do you apply two extensions?)

What did we use to build it? phpUnit 3.4.10 unit testing. Phing. Symfony YAML. Sismo. Subversion. Jira. And Trac.

Doctrine 2 Architecture

“Everything is an entity.” A lightweight persistent domain object, a regular PHP class. You don’t extend a base doctrine class. No final methods.

namespace Entities;
class User
{
  private $id;
  private $name;
  private $address;
}

Note use of namespaces.

EntityManager is central access point to ORM. Used to query for persistent objects. Employs transactional write behind strategy that delays execution of SQL to execute it in the most efficient way.

Tests ran against multiple databases. Sqlite, MySQL, Oracle, PgSQL, more to come.

Unit Testing

Impressive #s of test cases and assertions. Tests run in a few seconds instead of 30-40 seconds. Very granular, explicit unit tests, not many functional tests. Easier to debug. Continuously integrated by Sismo.

No, Sismo is not available yet, bug Fabien! (We have a pkcommit script of our own that doesn’t let you commit without a warning unless you pass symfony tests, which is a nice poor man’s version)

Database Abstraction Layer

Standalone package and namespace (Doctrine\DBAL).

Can be used standalone.

Much improved over Doctrine 1 in regards to the API for database introspection and schema management. (Can you alter a table in a more db-portable way wrt type names?)

Hopefully defacto standard DBAL for PHP 5.3 in the future like MDB was for 4.x

prepare($sql)
executeUpdate($sql, $params)
fetchAll... fetchBoth($sql, $params) - both assoc and flat...

DBAL Introspection

listDatabases
listFunctions
listSequences
listTableColumns($tableName)

Schema Representation

$schema = new \Doctrine\DBAL\Schema\Schema();  <- namespaces are nutty
$myTable = $schema->createTable("my_table");

Replacing diff tool with the ability to compare schema objects.

Doctrine 2 Annotations

/** @Id @Column(type="integer") @GeneratedValue */
private $id;

This gets parsed out. Alternative to schema.yml. (Lets you write everything in one place, so perhaps more convenient.)

Things to Notice

Entities no longer require extending a heavy base class
Domain model has no magic, is not imposed on by Doctrine and is defined by raw PHP objects and normal OO
Big performance impact
Easier to understand due to less magic. As Fabien says, “Kill the magic…”

(We’ve introduced a little magic here and there in our own stuff, maybe it’s less evil within a shop than without.)

Doctrine 2 *does* allow YAML.

Entities\Address:
  type: entity
  table: addresses
  id:
    id:
      type: integer
etc

You can do XML too

PHP “use” all necessary namespaces and classes

Recommended very long use statement, but it’s in the bootstrap function
Then require the Doctrine ClassLoader
Core classes, entity classes, proxy classes
Configure your Doctrine implementation by talking to the ORM configuration object

(A class called Configuration? Won’t this tend to clash with other stuff, like Symfony 2? I know about namespaces but Configuration means we’ll have to deambiguate every time)

Entity manager…

$connectionOptions = array('driver' => ... )
$em = EntityManger::create($options, $config);

Use a closure to lazily load the EntityManager…?

$user = new User;
$user->setName('Jon Wage');
$em->persist($user);
$em->flush();  <- Performance optimizes all that has gone before. Multiple row insert in MySQL? OooOooo

No more magic methods!

Inserting 20 records with Doctrine: way fast... compared it to raw MySQL... Doctrine 2 is faster because he's rolling a single transation around the 20 inserts compared to naive PHP that has no transaction calls.

(I didn't realize transactions had such a big performance benefit, I thought of them solely as a consistency and concurrency issue... without them MySQL must do an implicit transaction for every INSERT which slows you down.)

Doctrine blog piece on transactions and performance

DQL parser re-written from scratch. DQL parsed by top down recursive descent parser that constructs an AST. Generates the SQL to execute for your DBMS. DQL has a real BNF now and tells you what your errors are

(You could more easily teach it to manage a non-SQL backend now. Or accidentally de-optimize your SQL. Either way)

$q = $em->createQuery('select u from MyProject\Model\User u');
$users = $q->execute();

Cache Drivers

fetch, contains, save, delete methods similar to sfCache classes

Wrap existing Symfony, ZF, etc cache driver instances with the Doctrine interface

Slick ways to clear part of the cache:

deleteByRegex($regex)
delteByPrefix($prefix)
deleteBySuffix($suffix)

These are used with the result caching feature. (Which we should probably leverage in Apostrophe, and use the above delete methods to avoid inconsistency. This stuff has been backported to 1.2)

$query->useResultCache(true, 3600, 'my_query_name'); <-- cache key. These are easily purged fast with the above

New CLI

dbql:run-sql executes a manually written raw SQL statement or file. (We could make sfSyncContentPlugin db-portable with this perhaps... it's half of it)
orm:clear-cache clears query, result and metadata cache, has options to be more specific
orm:convert-mapping converts metadata information between formats. Convert between YAML and XML and annotated PHP.
orm:ensure-production-settings verifies that Doctrine is properly configured for a production environment, throws an exception if the environment is not suitable for production. (? What's suitable?)
orm:generate-proxies
Proxy objects are objects that are put in place or used instead of the real object, adding behavior to the object being proxied without that object knowing. Makes lazy-loading features possible. (Is this how you add multiple "behaviors" to one object?)
orm:schema-tool --update compares local schema to database and updates database accordingly, period. What about fancy migrations? "They don't exist yet but they will as an extension"

Inheritance

• Mapped Superclasses
• Single Table Inheritance
• Class Table Inheritance

A mapped superclass is an abstract class with no table in the database, it is a parent that can have multiple children, you get one table for each child class. "Concrete inheritance" in Doctrine 1, I believe.

Single table inheritance = column aggregation inheritance in Doctrine 1. A single table with the columns of all of the children. One of the fields is used to discriminate between types and hydrate the right class of object. (Will there be a prefix naming scheme of some sort to prevent column name conflicts when subclasses are written by multiple people? There isn't in Doctrine 1.)

Class table inheritance. This is new. Employee extends Person, BlogPost extends Node. MyCustomBlogPost extends BlogPost. Node is a base table with an ID. BlogPost has a foreign key back to node, MyCustomBlogPost has a foreign key back to BlogPost. Doctrine builds a join across all three tables.

Jon's example: a CMS with many node types. BlogPostCustom inherits from BlogPost inherits from node. BlogPostCustom has a blogpost_id field, which is a foreign key on blogpost, which has a node_id field, which is a foreign key on node. Again you still need a field to identify the right class to hydrate. And you need to know which subclasses exist so you know how many LEFT JOINs are needed.

(This is elegant, but is it fast? Going with Jon's CMS example, which seems inspired by Drupal, if you have ten node types on a page you have at least 11 classes in your join. In Apostrophe we use Doctrine column aggregation inheritance for slots, but generally serialize() things into the value column unless we need a foreign key. For the media repository we do use a refclass connecting slots to media objects, which is similar to what Jon proposes here. We'll be adding an API that lets you add additional joins of this kind. It's probably a win to avoid joins for simpler subclasses and use them only when there's a big payoff like ON DELETE CASCADE. Of course I'm prepared to be convinced otherwise by good benchmarks.)

Batch Processing

Batch processing by taking advantage of transactional write-behind behavior of entity manager... insert 10,000 objects with a batch size of 20, calling flush() and clear() every 20 objects. Calling clear() detaches all of the objects from doctrine. I can also detach single objects with detach(). If I don't detach them, Doctrine must keep a reference to them in case they change and therefore need to be included in the next flush(). So to avoid memory leaks in long running tasks one must detach or clear appropriately.

Query::iterate() method avoids loading everything into memory at once, bringing things into memory one at a time. Call flush() to execute your updates, then clear(). Note that you can change fields in your already-persisted object and call flush() and they save. save() is implicit and no longer exists as a method on an object. Remember, your objects are just objects now, and they do not inherit from Doctrine_Record anymore.

Bulk Delete

Same thing, you can iterate and then call remove() on things, and flush() does the actual work, call it at sensible intervals for performance.

Native Queries

You can now define an express mapping between a native SQL query and the fields of your objects to hydrate them exactly as you wish.

Proxies

Proxy objects seem to be wrappers for your original objects that come into play in a way that probably helps to implement stuff like what behaviors do in Doctrine 1, but this wasn't discussed at length. I'd like to know how one does NestedSet, Timestampable and the like elegantly with Doctrine 2.0.

Jon is thinking a stable version is 6-12 months away, possibly dovetailed with the next major releases of Symfony and Zend.

Code Generation

Jon was asked how you would add a phone number to a customer, or other one to many relation, if there is "no magic" in Symfony 2.0. It's clear how columns of a table are handled (a persisted Doctrine object's properties are grabbed and saved on flush() ) but relations seem to require some magic.

Jon replied that before the stable release there will be code generation available for such purposes.

Some people felt this amounted to an admission that Propel does things the right way.

An audience member asked Jon if he'd just reinvented the latest version of Propel. This is the most-tweeted comment of Symfony Live 2010 so far. But the answer is no.

Propel 1.5 resembles Doctrine today because Propel 1.5 borrowed much of the DQL query language for its new convenient, chainable query syntax. If Propel can lift concepts from Doctrine, then it doesn't make sense to knock Doctrine for adopting code generation if that has proven to be the right way to go.

What's more important is that, based on my reading of the what's new in Propel 1.5 documentation, Propel has not adopted the flush()/clear() technique that makes Doctrine 2.0 so fast.

Propel 1.5 seems to have caught up with Doctrine 1.3, no more or less.

Doctrine 2.0 does need to do a good job of replacing the behaviors feature it will apparently abandon, especially since Propel 1.5 has a number of behaviors (again imitating several seen in Doctrine). I look forward to hearing Jon articulate his plans for things like nested sets, sortable, and timestampable tables in Doctrine.

(Incidentally, limitations of Propel's nested set feature are one of the reasons why we didn't choose it for Apostrophe. A nested set without a level column can't be used to efficiently fetch children of a page only to a certain depth. This is a potentially fatal flaw in a CMS application with thousands of pages.)

He wrote the Symfony Internals chapter

Customizable, flexible, extensible

Extensible through plugins, events, and overriding entire framework classes

The bootstrap retrieves application configuration, then instantiates sfContext and calls the frontend controller

sfContext implements the Singleton design pattern. Has pros and cons: mostly cons. Testability issues. Responsible for loading factories.

Application configuration: frontendConfiguration extends ProjectConfiguration, shares methods and constructors. Most customization happens in ProjectConfiguration.

ProjectConfiguration crewates an event dispatcher, then loads plugins.

sfEventDispatcher is available as a separate component. Implements the Observer design pattern. An object can register interest in events.

autoload.filter_config event allows you to customize the autoloader configuration.

There are more than 900 plugins in the repository today.

settings.yml holds all framework leve settings, there are a lot of them. Actions, security strategies, I18N, debug bar, etc.

app.yml is for your use. config/app.yml is a project wide app.yml, we rarely use that.

Environments allow you to specify different behavior for different environments. Different database, mailer strategy, etc.

Config Handler

Not a well known part of Symfony. Parses and translates configuration files. Each configuration file has a handler. sfDevineEnvrionmentConfigHandler, sfDatabaseConfigHandler, etc.

sfContext instantiates factories. These are components that drive your application: logger, I18N, mailer, request, response, etc. Configured through factories.yml. This allows you to to subclass the normal implementations and let Symfony know that you want it to instantiate yours instead of the parent class.

In your app’s config/factory.yml.

sfFactoryConfigHandler converts config/factory.yml into executable PHP code to load the factories in question. It’s all in your cache.

You can override almost every component used by Symfony via factories (see the book).

Events notified include request.filter_parameters, routing.load_configuration, and context.load_factories. context.load_factories is the earliest event at which point database access is possible.

The front controller implements the front controller pattern. Grabs module and action from the request and issues a simple forward().

sfError404Exception forces a redirect to configured 404 handler. (How about exceptions to force redirect to the “login required” and “credentials not good enough” handlers? That would save effort too.)

sfGeneratorConfigHandler: instantiates a generator. Runs it. There is no step 3.

Controllers dir: you can override getControllersDir(), which gives you control over the controllers’ location. You can add any directory to the list or just replace it. (What class is this method in?)

The Action Stack

A FIFO stack. Holds every action that has been or will be executed. Access it through sfContext’s getActionStack(). (When do you have multiple actions on the stack? That sounds like internal requests, which are coming in 2.0.)

You can enable and disable modules through sf_enabled_modules in settings.yml and through mod_MyModule_enabled in module.yml. Modules disabled through settings.yml cause an sfConfigurationException to be thrown, use them for permanent disabling of modules. Those disabled through module.yml redirect to the “disabled module” action (sf_module_disabled_module/sf_module_disabled/action). Better for temporary disabling of a module.

The Filter Chain

Implements the Chain of Responsibility pattern. Configurable through filters.yml. Has a config handler too.

You can write your own filters to be part of the chain of filter objects that can intercept a request. You add it between security and cache.

rendering: ~
functional_test:
  class: swFilterFunctionalTest
remember_me:
  class: sfGuardRememberMeFilter

The rendering filter does nothing before calling the next filter down (but it’ll take action when that filter finally returns). A filter can execute code before the rest of the stack… but also after. So if you override the rendering filter you can add magic like autoloading of static stuff for the browser.

The secure filter implements credentials checks based on security.yml. Nice if the credentials are not dynamic and dependent on the object in question (aka “row ownership”).

The cache filter has two bits of logic: it can prevent the rest of the chain from executing. It configures the HTTP cache headers. And it fills in the cache before rendering.

The execution filter actually runs your action. Checks for cache, if no cache is found, executes the action. (The cache filter doesn’t do this?)

Execution workflow: preExecute(), execute via sfActions’ execute() and therefore executeActionName(), then postExecute()

You can return whatever you like from an action to trigger a template by name. SUCCESS, NONE and ERROR are predefined. (You can also return $this->renderPartial or $this->srenderComponent now, which is very useful)

Handling the View

This is sfView’s job. There are two ways of getting a view object, a custom sfView object for a specific action or a class name based on mod_module_view_class. (We have never overridden the view, we’ve always used templates)

sfPHPView loads core and standard helpers. Helper, Url, Assets, Tag and Escaping (sf_standard_helpers). Executes a view script (pure PHP) and decorates the result. Also handles some caching (partials caching?).

sfPartialView is responsible for rendering partials and components. Handles cache for partials and components.

Samples of custom views: sfTwigView integrates the Twig template engine by Fabien. sfTemplatingView integrates templating from the symfony components (the standard view?).

The rendering filter sends the response content through sfWebResponse’s send method (aha, the rendering filter does do something).

Question: why should we not use sfContext? And can we configure multiple databases for one application?

Answer (sfContext): the singleton pattern means you can’t easily change what singleton is being used, except with dependency injection which is not in Symfony 1. It tends to slow down unit tests to performance similar to functional tests because it is so heavy. (We at P’unk Avenue generally stick to functional tests especially since we care quite a bit what user is attempting to do what action at the model level.)

Q: can I avoid having five files called actions.class.php in my editor?

A: you could do one action per class.

Q: can you add listings for entirely new factories to factories.yml?

A: no, the list of factories to be instantiated is hardcoded. (But nothing is stopping you from loading a class name from app.yml and instantiating it, which is pretty much all factories.yml is doing. We do this to let you substitute alternative form classes etc. in sfDoctrineApplyPlugin.)

Q: What can you do by overloading the autoloader?

A (from audience): convince it to scan your own folders first. This allows you to override things that otherwise cannot be overridden, like core Propel classes. Poor man’s dependency injection. (I hear this will be oh so much easier in 2.0)

He’s using Zend Framework at the moment but has done a lot of good Symfony work in the past

“Backend development” photo of a zoopeeper holding up a trash can to an elephant’s rump very nice.

John worked with the NHS for a while, they were using Excel spreadsheets for complex workflows, web apps were a big change for the better

A big fan of the admin generator: “it’s super awesome”

Provides most common admin requirements out of the box (CRUD: Create, Retrieve, Update, Delete actions to manage more or less any type of information)

Can be extended to meet bespoke needs

What’s new since 1.0? “Completely re-written for the form framework”

Many-to-many relations work without further configuration (*cough* as long as you don’t do embedded forms)
You don’t have to expressly explain refClasses to the system
generator.yml is validated, finds errors for you
Batch actions (delete many easily) – wasn’t this in 1.0?
Less reliance on generator.yml, more DRY, the schema and form framework already spell out a lot.
More templates and actions to override
Separate configuration for ‘edit’ and ‘new’ actions in generator.yml
Adds a REST route for your module (doctrine route collection)

New PHP configuration file in your module: lib/newsGeneratorConfiguration.class.php

You still want to look in the cache for generated examples to override

First access it, then look at it: cache/frontend/your/module/etc

Held a vote: do you like configuration in PHP or in YAML? More and more he prefers PHP. I have come to like PHP because I’m not typing symfony cc all the darn time (well I wouldn’t be). I voted for PHP. Of course YAML is easier to sell to administrators who are not programmers.

Think requirements, don’t jump to use the admin generator. What do the users actually need to do?

(I’ve been on the other side of this sometimes though. “Oh this is a really simple module, I want to customize it a lot and I don’t need the admin generator.” Oops, now I have lots of filters and I’m reinventing the admin generator when I could have customized it)

In general, if it doesn’t feel like CRUD, you shouldn’t use it.

@jcleveley says it’s not good news to allow use by non-trusted admins, or very heavy use. But the admin generator is generated code, not metacode… performance should be pretty good actually. We’ve had good success overriding methods with additional security checks and you can also use security.yml

Admin generator 10 commandments:

• Understand the user’s workflow and customize admin to suit
• Think about security from the start
• Look through and understand the cached (autogenerated) PHP files
• Change table_method to reduce db calls. By default, on the list page, there are no joins being made. You should substitute a new table_method that does any needed joins to fix performance
• Use bespoke Form class for admin if you don’t want to just let them edit all fields of the model, you can set which form class in generator.yml
• Keep all form configuration in the Form class if you can, not generator.yml
• If you need to make changes to multiple admin modules – create a theme. (We have a nice one, take a look at the aAdmin theme in apostrophePlugin)
• Think about small screens and target browser. NHS had tiny screens and IE6. Be realistic about your client’s environment, not your favorite one
• Create functional tests – guard against regression. No excuses
• Maintain good MVC and decoupling practices. No HTML in the model, no queries in the partials (realistically you do wind up with some get() calls on objects to get their fields, but no heavy model code)

Major Admin Generator Points

1. Think about the URL. Clients don’t like:

application.com/admin.php

Try:

application.com/admin

Discussed web/.htaccess rewrite rules. His preferred method is a separate virtualhost that goes to an admin.php front controller.

(We at P’unk Avenue prefer not to have a separate admin controller and app, we prefer to customize the user experience to the user inside a single Symfony app, which makes the URLs even friendlier and the experience more contextual)

2. Dynamic MaxPerPage

People like a choice about how much information appears on one page of the list view.

Add a select box within the _list_header.php “records per page”
Override getPagerMaxPerPage

3. Adding relations

Fewest clicks for common tasks
Relevant data all placed together
Currently not directly supported by the generator
But: you can use mergeForm, embedForm, and sfFormDoctrine::embedRelation.

(embedForm has big problems. Many-to-many in the embedded form will not save properly, you must override updateObject and implement your own link/unlink stuff)

You don’t want to switch back and forth between two admin generators for, say, employees and phone numbers. You want phone numbers in the context of editing employees.

Apparently the solution is embedRelation in your form class configure method:

public function configure()
{
  $this->embedRelation('PhoneNumbers');
}

Make sure you hide the id:

$this->widgetSchema['employee_id'] = new sfWidgetFormInputHidden();

This lets you edit but it doesn’t let you add and delete. What to do?

There’s a symfony plugin that contributes delete and add features for embedded relations:

ahDoctrineEasyEmbeddedRelationsPlugin by Daniel Lohse

Has created embedRelations (plural) method

Gives you embedded forms with new and delete capabilities

4. Internationalization

Ha: demonstrated an xliff internationalization file for Vulcan

You tell the admin generator to use it:

generator:
  class: sfDoctrineGenerator
  param:
    i18n_catalogue: startrek

5. Tidy up filters

Filters work great but the default style is a bit off on a small screen (800×600 at NHS ouch!)

Fortunately there is a lot of CSS provided so it’s easy to move them to go across the top.

Again, peek at aAdmin for a nice example.)

6. Timestampable fields

Unset created_at and updated_at. Or make them static:

Override the template, or…

Create a plaintext widget

sfWidgetFormPlain widget (Stephe.Ostrow has submitted it for the formextra plugin, not in there yet; I wound up doing this for a client project too)

trac.symfony-project.org/attachment/ticket/7963

7. Pre-filter the list

list:
  object_actions:
    _edit: ~
    viewPhones: { label: Phone numbers, action: viewPhones }

Set filter attribute in user session. This is a bit complex. (Not certain how this is “pre-filtering”)

8. Row-level ownership

Only allow owners of objects access.
Example presumes sfGuard plugin and there is a user_id field as a foreign key on each object

protected function buildQuery() {
  $query = parent::buildQuery();
  $query->andWhere('user_id = ?', $this->getUser()->getId();
}

preExecute() {
  if the action is not new, check getUser()->isOwner($object)
  which looks for a user_id on the object, PHP is polymorphic
  so we don't care what class it is
}

(Our preferred solution is a userHasPrivilege module on the model class, which looks at the currently logged in user if no sfGuardUser object is passed. Our standard privilege names are ‘new’, ‘edit’, ‘delete’ and others as needed for that particular model. This is very extensible and allows the model to decide what is appropriate for a particular object)

Remove the user_id widget from the form of course, and in doUpdateObject (not updateObject ?), set the user id. Get the user id from the context -> getUser() -> getId()

Cute satan graphic pitchforking the sfContext call, you can inject the user into the form constructor instead

9. Custom filters

Find out who has a birthday today

addBirthdayTodayColumnQuery($query, $field, Value)

(Is this auto-sniffed or is there something in generator.yml?)

sfAdminDashPlugin. Joomla style admin dashboard with configurable admin navigation. Replaces the admin css. Manyally add header component and footer partial to layout. Nice if you don’t have time to bespoke. By Kevin Bond.

sfAdminThemejRollerPlugin by Gerald Estadieu

Looks stunning, jQuery, theme roller system, popup filters, abs in edit view, completely new admin theme

Extending Methods

Sometimes you want to just define an alternate CSS file in generator.yml.

You can also override templates and actions, but you can’t re-use that between modules (unless you move it to a theme). Can become untidy and hard for other devs to follow

So: create a theme. “Steep learning curve – PHP in PHP!”

mkdir -p data/generator/sfDoctrineModule/newtheme

Then copy the generator files from sfDoctrine plugin

[Screenshot of path. He's on Windows]

Extending admin events

admin.pre_execute: notified before any action is executed.
admin.build_criteria: helps you filter criteria used for list view. An alternative to overriding buildQuery.
admin.save_object and admin.delete_object

The future: “so, what does django do?”

You get a dashboard with all of your modules laid out. “Show related models” is more remarkable, you automatically get embedded forms for related models that let you manage connected objects. Object history. Delete related objects warning: warns you about what ON DELETE CASCADE is about to do to various objects.

Future: scope and features

Better support for embedded forms
sfGrid
Fulltext search
Save and list! Should act like save button in any other application!
Dashboard
Nested sets and ordering as standard features
Inherit from multiple themes. Right now you can only have one

(Symfony 2.0’s “everything is a plugin and a dependency injection container” philosophy should yield great things here)

Thomas Rabaix talk on internationalization

URL: routingModel: doctrine + propel
Form: sfForm

Most of the 18n is based on Prado framework
ICU data updated in sf1.3/sf1.4
You need

all:
  .settings:
    i18n: true
    default_culture: en

Culture detection

HTTP request -> culture in URL

If no culture in URL, culture in session

What about culture in the header?

Updates sfRouting, sf18n, executes controller

We’re not big on culture in URL in apostrophe since the session can do it but it’s nice to be able to bookmark.

sfContext::getInstance()->getUser()->setCulture('fr')

Notify the user.change_culture event (this doesn’t do the changing, it just notifies other things that care; why doesn’t setCulture do this for us?)

recipe_esarch:
  url: /:sf_culture/recipe/view

Example: /en/recipes/search => sf_culture = en

It’s built in, you don’t have to pay any mind to the culture yourself.

Or:

/recettes/recherche
/recipes/search

This is not built in, you would need suitable routes. Do we want i18n slugs for Apostrophe?

ysfDimensionPlugin

You can have custom config files per dimension, including a culture dimension

SF_APP_FOLDER/config/en/routing.yml

Con: duplicate route configurations. Pro: no performance hit from trolling through irrelevant routes

Showed a custom subclass of sfPatternRouting, would like to see that fly by again (:

Doctrine is succinct:

Page:
  columns:
    title: string(255)
		content: clob
		enabled: boolean

	actAs:
	  i18n:
	    fields: [title, content]

Our friend the clob appears

Default model culture can be accessed through sfDoctrineRecord::getDefaultCulture(), sfDoctrineRecord::setDefaultCulture()

Doctrine:

$page->Translation['fr']->title
$page->getTitle()
$page->title

Propel is a little cleaner

$page->getTitle('fr')

But I suspect you are usually setting the default culture to the user’s culture and then going merrily on

ORM never automatically left joins the translation table when building a fancier query. You must do it.

NumberHelper:
format_currency

<?php echo format_currency(1000.999, 'EUR', 'en') ?>: Eur1,0001.99
<?php echo format_currency(1000.999, 'EUR', 'fr') ?>: 1 0001,99Eur

format_date and format_datetime helpers are internationalized, we should be using them in preference to date

__() helper translates individual strings in any context

<?php echo __("My name is %name%", array("%name" => "Thomas"), "helpers") ?>
=> My name is Thomas

format_number_choice helper

<?php echo format_number_choice('[0] no file[1] one file[1, Inf]', $n) ?>

Probably didn’t copy this quite right but it is a very convenient way to echo one of several possibilities based on a variable, internationalized

sfNumberFormat

sfForm introduced by Alien pic everyone is a little terrified of it at times

sfWidgetFormI18nChoiceCountry, Currency, Language, Timezone, Date, DateTime, Time are all internationalized (although the date and time widgets are very very basic)

Culture is set in the controller, “never use sfContext::getInstance() in your form.” It’s not clear why not.

Form has a translation callback function through the sfWidgetFormSchemaFormatter class, setTranslationCllable. You can override this to change translation behavior. sf1.3/sf1.4 choice widgets extending sfWidgetFormChoiceBase automatically translate choices provided (so you don’t have to call __() yourself, and shouldn’t)

Translations can come from a lot of sources… MySQL etc. Generating translations is a bit complicated

So Thomas recommends mgl18nPlugin kindly open sourced by menugourmet.com

Can Apostrophe support culture prefix in URLs out of the box? Probably since it’s at the very beginning, a variation on our suggested routes would work fine. And we already automatically fetch the slots for the user’s culture.

Probably the easiest way to get a feel for Apostrophe is to go to our demo site and login. The user name is “admin” and the password is “demo”. Keep in mind that the content resets itself every hour on the hour.

If you want to get your bearings before you you do that, head over to the newly revamped Apostrophe site. For the very interested, you should consider checking out our extensive README which includes an Installation section, an Editor’s guide, a Designer’s guide and a Developer’s guide.

Now for some of the highlights.

Slots!

There are rich text, image, slideshow, video, button, PDF, Raw HTML, and other slots built in. If you have used Apostrophe before, you know that you can add a slot on a page, choose what you want to place and then reorder it. New custom slots can also be made for any project.

Slot Options

For some time we have wanted to be able to give you more fine control of a specific slot. For instance, what if you wanted to make the background color of a some text slots yellow, and some white? In the past that would have required two slots. Now, it can just be an option of the same text slot. In this screenshot, you can see that we allow you to switch between different types of slideshows. Make sure you try that out in the demo. It is probably our favorite new feature. We really sweated the details on making sure the option control was contextual. We hope you appreciate the effort.

There are many other new features and highlights, but for the moment I will leave you to find them in the demo (remember: user is “admin” and password is “demo”) or in the documentation. (We promise to highlight these in future posts.)

So, what does this mean to you?

Well, it depends. If you are a PHP developer with Symfony skills you can get started today. From our README:

Right now Apostrophe is best suited to PHP developers who want to make an intuitive content management system available to their clients. Apostrophe is very easy for your clients to edit, administer and maintain once it is set up. Right now, though, Apostrophe installations does call for some command line skills and a willingness to learn about Symfony. We are working to reduce the learning curve.

If you are a front-end developer and want to get started using Apostrophe, talk to us. We are in the process of working out a package where we set up a staging server with Symfony and Apostrophe running on it. We would train you on templating in Apostrophe and you should be up in running in a day or two.

If you don’t want to go that route, here is what we suggest:

Front-end developers who do not yet have PHP and Symfony skills but wish to set up an Apostrophe site by themselves should consider tackling the Symfony tutorial to get up to speed. It’s not necessary to complete the entire tutorial, but it helps to have at least a passing familiarity with Symfony.

If you just think Apostrophe is rad and you want one… like now, just drop us a line. We would be happy to design and implement an Apostrophe site for you. You would be joining some good company if you do.

Surprise!

We (more accurately Tom) decided to build an extra slot last night as a gift for making you wait an extra day. Isn’t that nice of us (him)?

If you are observant you may have noticed an extra slot in the screenshot above. So without further ado, we present the Feed Slot (commonly referred to as the RSS Feed slot, but it will take Atom feeds as well). Use this to bring in your Twitter updates or maybe our Apostrophe SVN code commit feed.

Feel free to thank us anytime. (We like expensive beer.)

This is really just the beginning. We hope you have an opportunity to check out Apostrophe. We would love to hear from you if you do. Comments below would be much appreciated (and read), as well.
__________________

Don’t forget to follow us on Twitter for on-going updates: @apostrophenow

If you are a developer, the Apostrophe Trac is the home of the open source community. On that site you can open support tickets, browse and contribute to the Wiki, and access and potentially contribute to the source code via svn (you’ll need to contact us if you want write access).

Also, please join the Apostrophe Google Group if you want to tap into the community of people using Apostrophe to get support and help.

We have decided to push back the release of Apostrophe 1.0 until tomorrow.

We are truly in the home stretch. Just hitting a few bugs, finishing up documentation and adding some instructional content to the demo site. To tide you over, we are publishing this screencast of how you add video to a project by searching YouTube from within Apostrophe. Note how we bring in associated metadata (url, title, etc). For the record, you can add video by pasting in embed codes from countless other services like Viddler and Vimeo.

Please tune back in tomorrow.

P.S. We are not blaming the snow for the delay, but did want to note that we are certainly under a deep white blanket here in Philadelphia.