It sounds fine, but rarely works out in practice; people are extremely reluctant to change running code, and even for new, green-field projects they are very reluctant to switch way from a language/version that they already know.

Changing existing, running code that “works fine” is not something that ranks high on any project’s priority list. Rather than applying effort to things that the managers thought had been paid for already, just to be able to upgrade to a newer release of a language or platform, they will decree that the developers should just stay on the old release “for now”. You can try to entice your users with great features only available in the new release, but it’s a gamble where you risk decreasing your user base for no clear gain for the language; cool, modern features cannot easily be weighed against the price of fragmented installation base in popular opinion, and you run the risk of getting a reputation for being an “upgrade treadmill” that requires constant effort to keep running when compared to more relaxed languages/platforms.

(Obviously, most of this doesn’t apply to projects written by hobbyists just for their own pleasure. However (here be flamebait…) PHP is disproportionally rarely chosen by hackers because it’s such a pleasure to write with in the first place.)

You’re underestimating the impact of backwards compatibility; your estimate that all active projects would migrate in 3 or 4 years is far too optimistic.

Suppose I’m a PHP developer. PHP has flaws, but I know how to work around those flaws – that’s part of the reason I get paid as a PHP developer. Now suppose that PHP 8 comes out and fixes those flaws, but it’s not backwards compatible. As a result:

• I have to spend time updating my code for PHP 8. That’s time that I could spend responding to customer requests, implementing new features, keeping up with the competition.
• Even after I’ve done this, there’s a good chance that I’ve missed some corner case or unforeseen compatibility issue, introducing bugs in my code.

Given this, there’s a strong incentive to never migrate to PHP 8, even if it is “better, clearer, more secure etc.” It’s estimated that there are still billions of lines of COBOL (!) – even though there are obviously much better technologies available, the cost of an update, combined with the risk of bugs, just doesn’t make it worth it.

Second, even if I decide to migrate my own code, any nontrivial app depends on third-party libraries, and there’s no guarantee that the third-party libraries will migrate. For example, Python 3 was released in December 2008, but Django (probably the leading Python web framework) didn’t have stable, production-ready Python 3 support for almost five years (see here and here).

You’re making a lot of assumptions about human behavior. If you change it too much, people will evaluate your competitors, since they’re going to have to spend significant effort switching anyway. For open source languages, people will just fork the old version.

Look at python for an example. 3.x has been available for four years, and still isn’t widely adopted. People try to use it for brand new projects, but I think you’re underestimating just how much code work is maintenance.

Of course, most people didn’t consider python 2.x to be “flawed.” They didn’t have complaints like php users. Php is in a much more precarious position, because a lot of people only stick with it because of its large existing code base. If you lost backwards compatibility, a lot of people would take the opportunity they’ve been waiting for to move to python.

For any language other than PHP I’d say, yeah, that absolutely makes sense! That’s exactly what Python is doing with the switch to Python 3.

However, the problem with PHP is that there are too many flaws with the language design itself, thus what you’re calling “PHP 8” would be a completely different language. And if you’d have to switch to different language, why would you stick with new PHP, rather than any of currently existing and stable alternatives?

Also PHP community is extremely slow to adapt anything new. Just look how long it took to get rid of register_globals. It’s been known to be a security risk since year 2000. It’s been only finally removed 12 years later. Another example, when PHP5 was introduced, it was huge improvement over PHP4, yet community did not adapt it. I took 4 years and massive actions such as GoPHP5 to jumpstart the adoption. And that even didn’t have significant amount of backwards incompatible changes.

Disclaimer: I manage a ColdFusion user group.

ColdFusion suffers the same problems: loved by many, despised by many. In addition, tons and tons of FUD based on pre-Java versions. ColdFusion 10 came out last year, is a huge seller and last week I signed up to test the pre-release of version 11. Also, there are two major open source alternatives, one is backed by JBoss.

There are TONS of new functions in CF10 that I would love to implement, but migrating from CF 7 or 8 can be difficult depending on the size of your code base, the number of upcoming projects and the resources you have to regression test everything once you’re on the latest version. I’ve run into a number of minor syntactical differences between 8 and 9, as well as edge cases where the code doesn’t compile the same way. Once found, I documented them into our Coding Standards so they aren’t used in future projects or by new developers.

That said, if ColdFusion 11 (or any programming language) were to completely deprecate certain functions and syntax, the level of effort to find and replace functionality could be enormous. The testing effort could be gargantuan. Will companies pay their developers, QA and project managers to find, replace and test all of those deprecated things? Doubtful.

If the latest version of a language is backwards compatible, but introduces a boost in performance with no code changes (CF9 is about 30% faster than CF8 and CF10 is much faster than CF9), who cares about changing function calls if they still work?

As a company, we have to worry about pleasing our clients and delivering their needs in order to bill services, build the business and recruit more clients.

FWIW, I would love to get us onto the latest version of jQuery at some point, but since certain functions were deprecated a few versions after what we use, and given the volume of JavaScript we have in the system, I don’t know how we’re going to pull that off.

There’s a tradeoff here; some bugs REALLY need fixing, but some things can’t be changed without breaking someone’s code somewhere. I seem to remember someone stating as a “rule” that every bugfix will break someone’s project, no matter how obscure or obviously-broken the bug is, someone will be using it for something. Such is the nature of programmers.

This is (to my mind) the difference between major releases, minor releases, and revisions. As a general principle:

• Major releases are assumed to contain breaking changes.
• Minor releases may change behaviour slightly.
• Revisions should be pretty much cross-compatible.

For example, if I’m writing something in v2.3 of a language, I wouldn’t expect to notice any difference if I upgrade to v2.3.2. If I upgrade to v2.4, then a few things might change – small syntax tweaks, some functions behave a little differently so I have to tweak logic, etc. If I upgrade to v3.0, I would not be surprised if it broke entirely – functions deprecated or missing, operations not supported or changed so much that I can’t just tweak it back into line, I actually have to rewrite some functions to account for the new changes.

Edit:

Steve Vance’s paper Advanced SCM Branching Strategies has this to say:

Typically, there are two to three levels of release, named by numbers connected with periods (e.g. 1.2.3). […] In this structure the first number is associated with a major version, indicating that it has significant feature and functional enhancements from the previous; there may also be significant incompatibilities that require migration. The second number represents a minor version, which contains lesser feature and function enhancements, a significant number of bug fixes, and no incompatibilities. The third number refers to a patch level, signifying almost exclusively a collection of bug fixes; no feature or function enhancements and no incompatibilities are allowed between patch levels.

The only alteration I’d make to this is the aforementioned principle that programmers often find ways to “use” bugs, so a minor version with “a significant number of bug fixes, and no incompatibilities” might be difficult, because it’s likely that those bugs will either break something that used them, or will cause a workaround to become unnecessary and start causing problems.

It really depends on what’s the target of the language – what types of applications are intended to be built with the by the language.

For example, ignoring Android, Java is mostly used in large enterprise systems and middle-ware; these types of applications tend to become very large both in size and in time. This has some implications; imagine a system with 500K+ LoC on which worker 50+ engineers in the development phase. Typically this type of system enters in maintenance after this with say 10 developers; now if the language changes and the changes are not backwards compatible the project cannot be easily migrated to a new version because the programmers that wrote some parts are gone and nobody wants to touch it. This is the smaller problem, the bigger problem consists in the fact that is kind of expensive to adapt a 500 LoC application to new language constraints. For example if generics weren’t implemented with type erasure and List list = new List(); wouldn’t compile millions of lines of code would need to be rewritten – which is at a great cost.

On the other hand PHP tends to be used on the web for simpler applications; usually it is developed by a single programmer or a small team. The idea is that the developers kind of know the whole project pretty well can can integrate language changes more easily. Also it’s purpose is to build a site very fast, and the faster the better so if a new language feature can do this better then it is implemented even at some backwards compatibility costs.

One can argue Microsoft performed a similar change with ASP.NET (as successor to classic ASP) or with VB.NET (although they made so many concessions with the latter that most of the benefits of “rebooting” the language were lost).

Anyway, if anyone remembers the nightmare of Migrating VB6 code to VB.NET even with the assistance of a migration tool, they’ll agree right off that language migration tools don’t work very well for major language updates.

It might be possible to move the platform forward but, you should still provide support for “deprecated” APIs through at least a few revisions.

A lot of the “flaws” people scream about in popular programming languages aren’t, they’re things the screamer’s favourite toy of the day has that that language lacks, THEREFORE that language is fundamentally flawed because it lacks that.
The next hype comes around, the language is suddenly flawed because it doesn’t adhere to that hype.

The lack of closures in Java is a classic example. That’s not a flaw in the language at all, and changing the language (as is sadly on the agenda) to include them will IMO fundamentally cripple it or at the very least make it a lot harder to read and comprehend.

What all too many people lose sight of is that each language has its strengths and weaknesses and that trying to create something that combines the strenghts of everything while avoiding every weakness will only create an utterly unusable monster that’s good at nothing, incredibly unwielding, impossible to use effectively.

Add, as others have pointed out, that backwards compatibility is vital to retain existing users, many of whom are NOT going to spend the thousands of hours and millions of dollars/Euros to retrofit their million line codebases to whatever you think is “better” than the version of the language they’ve been using for years, and you’ve a host of very good arguments to leave well enough alone and if you want to play with some new overhyped idea that’s supposedly the next “java killer” you’d best play with that toy rather than scream that “Java iz ded” unless it “gets fixed” to be a clone of that toy.

I would suggest that newer versions of a language should strive to ensure that 99.99999% of code which compiles in both the old and new versions of the language should work identically in both unless it’s deliberately designed not to do so, and that most of the time when the new version refuses code that compiled under the old version, it will be because the code was–at best–dodgy, and should have been written in a different way that would compile under both the old and new compiler.

For example, if I were designing a new language similar to Java or C#, I would forbid implicit type conversions in some contexts where those languages allow them. As a simple example in C#, given

int someInt;
double someDouble;

the expression someInt.Equals(someDouble) is guaranteed to return false, regardless of the contents of the variables. It compiles because double can get converted to Object, and int has an Equals overload for that type, so the compiler does the conversion and makes the call. Were I designing a new version of C# and the .NET Framework, I would have it forbid the boxing conversion since it can’t possibly do anything useful. It’s possible there’s some program which does such a comparison in a way which is useless but harmless, and having the compiler reject such code might break that program, but correcting or removing such useless code would be an improvement.

As a slightly less clear example, assume

float f=16777216f;
int i=16777217;

and consider the expression f==i. It’s possible that some code does float/integer comparisons and works correctly, but the code should be rewritten as either f==(float)i, (double)f==i;, or (double)f==(double)i; [int to double promotion is lossless, so the latter two would be equivalent]. Some code which directly compares float and integer values may always deal with numbers which are sufficiently small that float and double comparisons would behave identically, but a compiler generally can’t know that; code should make clear what kind of comparison is needed, rather than hoping the rules of the language will match the programmer’s intention.

It’s best to never break backwards compatibility.

Microsoft replaced the VB6 programming language with a new language that totally broke compatibility. So even today the 16 year old VB6 is still more popular than the dotNet version (Tiobe index August 2014). And Gartner estimates there are 14 billion lines of VB6 code still in use.

In 2014 Microsoft have again had to announce they will not update or open source VB6 despite demands from the Visual Basic programming community.
But they have extended support of VB6 until ‘at least’ 2024, and it works fine on Windows 7 and 8. That will be over 26 years of support for the same version of VB6.

Why should existing working software have to be re-written, even Microsoft never “updated” Office to use dotNet ?

There are a few different problems with breaking backwards compatibility. Some of the problems stem from the fact that most programming languages are also platforms (interpreters/runtimes), other problems stem from an assumption of human nature.

A. Code written in an older versions wouldn’t get the benefit of new releases that improve performance, security, or features. You could mitigate this problem by supporting multiple major versions of the compiler/interpreter, but that is a huge resource drain (ie its expensive or takes a long time, and is a pain in the ass).

B. Code written for the newer versions may not be compatible with code written in older versions. You could work around this by having an interpreter/compiler that can handle multiple major versions of the language, but this is more of a pain in the ass than supporting separate interpreters/compilers (the workaround for A).

C. Major changes, if they happen too often/quickly also simply make the language harder to use, since you have more to learn and unlearn. Changes to a language might push people over the edge to switch to a new language, or might cause people to continue to use outdated versions of the language and just never switch to the new version (like has happened with python). Then again, the changes may also attract new users and excite old ones.

D. New documentation needs to be kept and maintained. Its always a rather confusing experience to look up things on google and find that you’re reading the docs for a different version than you’re currently using.

By and large, if you create a programming language where external modules don’t have to care what version you’re using, breaking backwards compatibility for the right reasons (to fix major flaws in the language) is almost definitely the right thing to do. Its likely that the major reason this isn’t done is that programming language designers overestimate (to contradict someone else’s answer) the costs of breaking compatibility, especially early on. The fact is, the problems of breaking compatibility can be worked around or powered through by users of that language. And this doesn’t only hold for programming languages; this applies to APIs, user interfaces – really any interface in any situation.

Facebook annoys the hell out of people when it changes its UI, or its developer APIs. In the past, its made the platform difficult to work with. In some cases, APIs simply stopped working out of the blue. But people kept using it, and now the APIs and UIs are leaps and bounds better than they were 5 years ago. People will complain about change whether its good or bad for them, but that (complaining) is not a good reason to forgo that change. Unfortunately, programming language developers use this as a reason to keep the problems of their language intact.

So another couple reasons for languages not making breaking changes to improve themselves are:

E. Language developers think their users fear of change is a good reason to stagnate their language

F. Language developers kind of liked their language when they made it, and they probably think its just fine with its flaws.

G. Languages as they grow older usually cease to have a small core set of developers, and turn into more committee-built beasts. This means decisions about those languages are slow and often conservative and uncreative.

H. The last reason is that some breaking changes require significant re-evaluation of design decisions made for the interpreter/runtime. Sometimes improvements to the language simply require too much work to be feasible. I would guess this is a rarer problem than most tho.

Often, language designers aren’t necessarily tool designers, and so they don’t think of good solutions to this problem, or they don’t execute them well. Here are some solutions I can think of to solving the breaking-changes problem:

1. Deprecate things well ahead of when they’ll be removed.

2. Provide a good, standard converter tool. Python provided the 2to3 tool, but it wasn’t well advertised, didn’t come standard with python 3 as I recall, and didn’t even work very well (I remember having to manually go through programs generated by 2to3 to fix issues it didn’t fix). This converter tool could even be automatically run if your compiler/interpreter detects an older version. What could be easier?

I don’t know if that’s a problem for PHP code, but in many languages old, legacy code is never updated after years or, sometimes, even decades, because it works, is critical for the business running it and is too big (say millions of SLOC), so it would not make sense to rewrite it. That’s a reason why java made backward-compatability an almost religious issue, despite know old problems, especially in libraries (even if they are easier to update). I guess a lot of code from the Linux kernel was not updated for decades too, despite the adoption of standards like C99 and C11.

Even in languages that are less “entreprisey”, breaking old, functional code can be a problem. That’s what happened with Python 2 -> 3. A whole bunch of libraries and system scripts were stable and not maintained anymore, not because they were abandoned but because they were stable and doing their work. Adapting them takes a few years. So, as a developer, you cannot necessarily jump to python 3 if your favorite library hasn’t made the move yet, so your own code will not work under python 3 either, thus resulting in community fragmentation.

The problem lies within the backwards compatibility issue. Most PHP scripts I execute are running on an older RedHat server. If I were to use the newer version of the language for future scripts, then I would have to update PHP on this server — and run the risk of having my older scripts broken/ having to take hours to rewrite all the old code with the new standard. Plus, all my developers are used to PHP reacting a certain way (whether that way is ‘broken’ or not). If it no longer reacts that way, it could be a major hurdle for productivity, as developers may have to basically re-teach themselves PHP.