Why PHP 6 Never Shipped and PHP Skipped to 7

PHP went from 5.6 to 7.0 in December 2015. If you started with PHP after that, you might not know there ever was a PHP 6. There was. Core developers worked on it for about five years. Its lead developer spoke about it at conferences, and publishers sold books about it. Then in March 2010, the project was shelved, and in 2014, PHP internals voted 58 to 24 to skip the number entirely. One small piece of it is still in PHP today, and as of this month, internals is talking about removing it. Here's the story on what PHP 6 was supposed to be, why it failed, and why the next version was called 7. The Problem PHP 6 Was Meant to Solve# In PHP, a string is a sequence of bytes. PHP doesn't know or care what encoding those bytes are in. That's fine for plain English text, where one letter is one byte. It stops being fine as soon as you leave ASCII: echo strlen('café'); // 5 In UTF-8, é takes two bytes, so strlen() counts five. Functions like substr() and strtoupper() have the same problem, since they work on bytes, not characters. Cut a UTF-8 string in the wrong place and you get broken text. We worked around this with extensions like mbstring and its mb_ functions. But the language itself had no idea what a character was. In August 2005, Andrei Zmievski put it bluntly in his design document for fixing it: As successful as PHP has proven to be in the past several years, it is still the only remaining member of the P-trinity of scripting languages - Perl and Python being the other two - that remains blithely ignorant of the multilingual and multinational environment around it. The Plan# According to Andrei's later talk on the project, he launched the Unicode effort at Yahoo in February 2005, with a small group from Yahoo, Zend, and the PHP community. The work was "initially slated for 5.x," but the changes grew big enough to need a major version. His design document, posted to internals on August 10, 2005, laid out the approach: ICU. PHP would use the International Components for Unicode library for Unicode support. UTF-16 inside. Unicode strings would be stored as UTF-16 internally, because ICU used UTF-16 for its own processing. A switch. A unicode_semantics setting would turn Unicode mode on for a whole site. Two kinds of strings. Text would be Unicode by default, and raw bytes would get their own binary type. A b prefix, as in b"...", would mark a binary string literal. The design document explained the UTF-16 choice: While being less memory efficient for basic ASCII text it simplifies the processing and makes interfacing with ICU easier, since ICU uses UTF-16 for its internal processing as well. That decision would come back to haunt the project. The next day, Andrei warned everyone that the Unicode code was about to land in the main branch: The warning goes into full effect now: do not commit anything to Zend/, main/, or ext/standard/ or I will hunt you down and make you memorize the entire contents of Unicode Character Set. In binary. The first "Unicode support" commits went into php-src late on August 11, 2005. The day after, Rasmus Lerdorf started a wishlist thread for the new version: Since we are breaking a lot of stuff in 6.0, at least with Unicode_semantics=On I am wondering if it may not be time to break some more stuff and do a bit of spring cleaning. The Paris Meeting# In November 2005, core developers met in Paris to plan PHP 6. The group included Rasmus, Andrei, Derick Rethans, Zeev Suraski, Dmitry Stogov, Marcus Börger, Wez Furlong, and Jani Taskinen. Derick published the minutes. PHP 6 got a long list of decisions: Unicode mode would be controlled by a php.ini switch that defaulted to on. Extensions that weren't updated for Unicode wouldn't load in Unicode mode. register_globals, magic_quotes, and safe_mode would be removed. An opcode cache, APC, would be bundled. Namespaces, late static binding with static::, and a 64-bit integer type would be added. The minutes also recorded an early warning sign: current micro benchmarks show that the Unicode implementations of some of the string functions are up to 300% slower, and whole applications up to 25% slower. Building It# Work continued in php-src's main branch, which was now PHP 6. Andrei renamed the switch to unicode.semantics in 2006, and Dmitry committed namespaces to the PHP 6 branch in 2007. PHP 5 got a small piece of the future too. In PHP 5.2.1, released in February 2007, the NEWS file listed "forward support" for the b string prefix and a (binary) cast. The idea was that you could start marking binary strings in PHP 5 code so it would be ready for PHP 6: $bytes = b"\x89PNG"; $data = (binary) $input; In PHP 5, a b string was an ordinary string, and (binary) was an alias of (string). They were placeholders for a version that hadn't shipped yet. Over time, developers started asking whether a Unicode on/off switch made sense at all, since code would have to work both ways. In May 2008, Andrei removed it with this commit message: Crossing the Rubicon: remove the unicode.semantics switch and default unicode mode to On. PHP 6 was now Unicode, full stop. PHP 5.3 Takes the Features# Meanwhile, PHP 6 was taking a long time, and the other features planned for it were done. In September 2007, Ilia Alshanetsky proposed a feature list for a PHP 5.3 release. The first item was "Backport the namespaces patch for PHP 6." Developers voted in favor of backporting namespaces, late static binding, and __callStatic from PHP 6. PHP 5.3.0 shipped on June 30, 2009, with namespaces, late static binding, closures, goto, and the short ternary ?:. It also brought in intl, an extension that wrapped ICU for things like formatting dates and numbers by locale, and worked with PHP 5. That left PHP 6 with one big feature. A slide in Andrei's talk summed it up: PHP 6 = PHP 5 + Unicode The Books# While all of this was happening, publishers were printing books about PHP 6. PHP 6 and MySQL 5 for Dynamic Web Sites, by Larry Ullman, from Peachpit in December 2008 PHP 6 and MySQL 6 Bible, by Steve Suehring, Tim Converse, and Joyce Park, from Wiley in 2009 Professional PHP6, from Wrox in 2009 Beginning PHP 6, Apache, MySQL 6 Web Development, from Wrox in 2009 None of the versions in those titles ever shipped. MySQL 6 didn't either. It's easy to blame the authors, but the 2014 naming RFC pushed back on that. It said PHP 6 was "taught to users as a done-deal, including detailed explanations about features and behavior (by php.net developers, not 'evil' book authors)." Andrei's own slides list "Spoke at multiple conferences about the project" among the things that went right. Rasmus saw it differently in 2010: People who have written books or articles about PHP 6 inferring they knew what the final state of PHP 6 would be were misguided. We never got to the point of a final feature set much less a release date. Why It Stalled# The core problem was the choice of UTF-16. The outside world mostly speaks UTF-8. Files, databases, and HTTP all use it. So every time PHP 6 talked to anything outside itself, it had to convert. Pierre Joye explained in 2010: The win we may have from using UTF-16 is almost completely lost by the conversions cost. Stas Malyshev listed performance and compatibility as the two biggest problems. Many extensions people relied on, like mysql and pcre, didn't handle UTF-16 well, and "inflated memory usage hurts scalability a lot." Existing apps could break in strange ways, including cases "where your passwords suddenly stop working because there's an extra recoding step in some md5() call." There was also a people problem. Converting every extension to the new string types was slow, tedious work. Johannes Schlüter, the PHP 5.3 release manager, wrote that it led to "many contributors not willing to use 'trunk' as their main development tree but either develop using the stable 5.2/5.3 trees or refuse to do development at all." Andrei was candid about it in his 2011 talk, "The Good, The Bad & The Ugly: What Happened to Unicode and PHP 6." His slides said there were "Probably <10 people who understood the intricacies of the Unicode and ICU," and that "People were bored converting large chunks of already working code." Looking back, he said he "Would probably choose UTF-8, if started over." March 11, 2010# PHP 6 ended with an argument about output buffering. On March 11, 2010, Jani Taskinen merged a large output buffering change into the stable PHP 5.3 branch. When people objected that it didn't belong in 5.3, he replied: Yadda yadda, it's a bug fix. Just call next one 5.4 like it should be. Derick's response was short: "No, put that stuff in 6!" A few hours later, Jani created a PHP 5.4 branch himself. The commit message was "Hello PHP 5.4, open for all new stuff." That forced the question. An hour later, Rasmus sent an email with the subject line "PHP 6": Ah, Jani went a little crazy today in his typical style to force a decision. The real decision is not whether to have a version 5.4 or not, it is all about solving the Unicode problem. The current effort has obviously stalled. He continued: We knew the Unicode effort was hugely ambitious the way we approached it. There are other ways. Rasmus proposed moving the PHP 6 code to a branch, moving development back to the main line, and looking for "lighter and more approachable ways to attack Unicode," pointing to mbstring and ext/intl. He also ruled out running two development lines at once again: I refuse to go down the path of a 5.4 branch and a separate Unicode branch again. Jani was frustrated by how decisions got made. In another message that day, he suggested renaming the PHP 6 branch to THIS_IS_LOST_CAUSE, and wrote, "I'm just sick and tired of the cloak and dagger style and secret meetings and committees." On March 16, Derick did the renaming. He announced: I've just renamed the 5.4 branch to THE_5_4_THAT_ISNT_5_4 and moved trunk to the branch FI