Unfortunate. I've been following the work on it occasionally.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
I think this is a major issue with requiring a PEP for what's a performance optimization.
JITs are big nasty things that by their nature have to do a lot of things. You are basically saying "Give us a PEP for a new python interpreter and compiler".
What they do is fuzzy by their nature because optimizations are very rarely universally beneficial. It's a game of measuring, looking for regressions, and tackling the biggest problem next for whatever sample applications you have available. You always run into the risk of overfitting on top of all that (You may have made django faster, but you've made flask slower).
This is the sort of thing that, IMO, should not go through the PEP process. It's an implementation detail for the runtime.
Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements. They'd never get done. Or if any javascript engine needed ECMA acceptance before they could do a new JIT optimization.
Does not have the same level of requirements being asked of the Python JIT. Further, it's well understood that this JEP will "live on main" for quite some time in preview before it's accepted and merged.
What the JEP does, instead, is simply describe the feature and potential risks.
The vast majority of JVM optimizations do not get a JEP.
Fair, I don’t argue with that, I’m not knowledgeable about Java’s processes. Just one thing:
> The vast majority of JVM optimizations do not get a JEP.
In Python too. Still, the JIT really is just a really big thing. I don’t know if there could be a better way to do it, as either 1. You develop it off-branch and… make an enormous PR when you’re done, with risks of getting rejected anyway? Or 2, you allow more experiments on main like Rust, but then you end up with hundreds of minor half-maintained nightly-only features, possibly conflicting with each other, and eventually go through bureaucracy to remove them anyway.
I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.
Dunno how much investment is needed into language optimization anymore.
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.
This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.
Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.
With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).
[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.
[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.
It's unfortunate that a technically exciting new feature is being killed partly because of what is essentially bureaucracy (and the lack of the developer's personal bandwidth). On the other hand, it's pretty safe to say that the strict adherence to the PEP policy and all of the bureaucracy that comes with it is what makes Python a great language, for better or worse.
Not to say that the technical objections were without merit, they likely very much weren't, and the debugger and instrumentation support and maintenance work sounds fairly painful on its own.
I think it would be a mistake to conclude this has anything at all to do with bureaucracy as a matter of inanimate policy that the SC was helplessly in thrall to. Bureaucracy was the tool the SC used to demonstrate that this feature lacked many important properties that they cared about, as mentioned: "maintenance, compatibility, and ... clear success metrics". For whatever detailed reasons, the PEP was withdrawn after it couldn't convincingly demonstrate those things to the satisfaction of the critics on the SC.
Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.
(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)
The problem is that clearly documenting all details as required in the PEP is at odds with complicated new features like a JIT where all the details can’t be known and have to be decided on as they are realized. Process / bureaucracy is at direct odds with experimentation.
But they weren’t asking for all the implementation details. They were asking for a clearly stated scope, success criteria, support plan, etc.
As they themselves pointed out, the problem isn’t that there’s anything wrong with an open-ended experiment. It’s just that open-ended experimentation doesn’t belong in main, of all places. They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
The scope of a JIT is the entire python runtime. Not at the beginning, it can be more narrow, but certainly it can evolve to cover everything. JITs touch everything.
> success criteria
Impossible to measure. JITs don't universally make things faster and unless the SC was willing to accept "some programs will see around x% improvement" then it would be impossible to actually give them the criteria they want. There's a reason C++ compilers and their optimizations aren't defined by the ISO committee.
> support plan
Like most JITs, support for a JIT is an ongoing and ever evolving process. What's needed depends entirely on what's needed today. You usually need more than 1 or 2 dev working on it, which may be the issue here.
> It’s just that open-ended experimentation doesn’t belong in main, of all places.
Very strong disagree. That's simply the nature of performance optimizations for JITs. They belong on main because new optimizations and heuristics will always be generated. The role of doing a compiler is a long term experiment in performance, seeing what your runtime is currently doing and how it's being used and tweaking it to work better.
pypy isn't in a worse place because it's been running a longterm performance experiment.
> They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
None of the things mentioned actually do that, other than perhaps a support plan. There are plenty of critical infrastructure projects who all can manage to develop JITs. Python is the special snowflake that has decided to do JIT development incorrectly.
This is a runtime optimization/implementation detail. It's foolish to send it through a committee approval process. That's why you won't be able to find a single language with a JIT or advanced optimizations that has do it by committee.
And yet, not introducing unnecessary chaos into the CPython project remains important.
Perhaps these priorities were always irreconcilable.
Perhaps that’s why the existence of competing Python JIT projects that weren’t trying to commit directly into CPython’s main branch kept coming up in the conversation.
In that case I can understand why it's not a fit for Python. The risks you're describing are more what a brand new language would face: universal scope, impossible to measure success, no support plan... sounds like a nightmare in a mature product. If that's actually what it means to add a JIT to Python, the SC is doing its job by steering Python around this quixotic task.
> This is a runtime optimization/implementation detail.
I view JITs as a critical architectural component. Like you said, it impacts the whole system. Optimization and implementation details don't usually do that.
The problem is that “Python” the language includes native poorly isolated FFI interfaces exposed by CPython. So the language isn’t “Python” but CPython. That’s why alternate implementations like PyPy haven’t seen adoption despite having a very mature JIT while languages like JavaScript have JITs that put CPython’s performance to shame.
> Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
Bureaucracy doesn't definitionally mean Kafkaesque. It's popular to hate on it, but good or neutral bureaucracy does exist.
It doesn’t have to mean kafkaesque, but it does mean governance through layered hierarchies and fixed procedures.
In this case a single committee directly engaged with both core developers and the community, through a fairly open process, explicitly bent several of their own official rules to suit the occasion, and indicated cases where further flexibility might be needed depending on how things develop.
I would call that a pretty stellar example of non-bureaucratic governance.
Large codebases are bureaucracies of their own, that is, the complexity of extra layers starts to bog things down at some point.
In this case, it seems like the human bureaucracy made wise and measured decisions to contain the software bureaucracy. They made reasonable requests to clarify the scope and impact of a major, major core feature. The proposers didn't feel like that was doable, so a complexity nuke was avoided.
There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
>There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
And all of them are lacking in different ways, we ALSO, want faster pure Python.
> we don't feel we have the personal bandwidth to carry the current version forward
Are the JIT developers the same people who were working on Faster CPython at Microsoft? No surprise that they don't have the same amount of energy to dedicate to the JIT now that it's gone from a full-time job to a hobby project.
Python is a mature production language with a lot of users. Even before it was huge - like 20 years ago when I was in college - getting a small backwards compatible API change into a python extension module (gnu readline in my case) required convincing GvR personally that the API change was necessary.
Every JIT attempt I've seen for python just tries to cut too deep in the first pass, and scope themselves too large.
I believe any successful attempt at optimizing python performance from JIT will FIRST start with a 100% semantically compatible hidden type model for optimizing object property access and method dispatch.
The single biggest performance delta in most dynamic language jits come from that, not from optimizing guards off of bytecode operations or from compiling to machine code. It's from converting hashtable lookups into constant-time guard + offset lookups.
The hidden type model is also a hard nut to crack. It needs to perserve dict semantics which have very dynamic behaviour via the various `__*__` methods, and handle exotics scope objects since scoped name lookups are the most common ones which have their own exotic behaviours (I remember python providing some ability for callee frames to inspect caller scopes in some limited ways).
I haven't kept up with python development lately, so thanks for pointing that out. I looked this up and it does seem like some sort of property access is being optimized. What's not clear is what the hidden runtime type model is being used for this, or if it's a lightweight tracking that captures some subset of property accesses.
Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.
The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
> Optimizing for this would require a tree of hidden map objects much like JS JIT engines use.
I don’t really think that’s true or even that relevant , at least in context of classes (especially with slots - the class, by definition, _is_ the “hidden map”) and not generic dicts.
> The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement.
Pretty sure the motivation was exactly the opposite - that a simple “if a LOAD_ATTR encounters the same class with slots N times in a row, swap it out for LOAD_ATTR_SLOT with inline cached slot index, and swap back to generic one if something changes” is way easier (both to implement and reason about) than any alternative that’d interact with the object model.
(Also it means you skip having to branch between a slow and fast path at all)
> you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Sorry, that sounds a bit vague and not too different from what CPython already is doing.
Ah, the `__slots__` feature makes it make a lot more sense.
> Sorry, that sounds a bit vague and not too different from what CPython already is doing.
With slots being available, I can see where I was misled. I went looking for a type structure that modeled the dynamic properties on an object which would be used as a typeguard, but the class itself functions as that if you're restricting the optimization to slotted properties.
(Note that this optimization can also trigger for plain classes (that’s LOAD_ATTR_INSTANCE_VALUE), it just needs a bit more guards and is only useful if you don’t abuse instances being dynamic objects. I just used slots as the most obvious example, but there are lots of different op specializations since like 3.11)
It still blows my mind that the big "we're breaking backwards compatibility Python 3000" release didn't do anything to lay the groundwork for this, or any of Python's other structural problems.
The problem was he exposed too much of the implementation into the language itself, destroying its ability to be a simple reference implementation that others could re-implement.
At the time, Python had a clear purpose as an easy to learn "glue" language between other, perhaps more performant languages. There was no reason to make Python as fast because that would muddy its purpose by introducing compromises and complexity. For better or worse Python sat pretty well in its niche.
Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.
A sad day for CPython but it doesn't have to be a sad day for Python. There is a fully maintained implementation of Python 3.13 that has an advanced JIT compiler. Once your Python warms up it can be 4x faster or higher (depending on what your program is doing). That implementation is called GraalPy and it builds on the JVM:
A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.
It has other interesting features
• Scalable pauseless GC
• Single binary distribution of Python programs (like Go)
• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.
In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.
(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)
So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).
This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.
Disclosure: I sit next to some of the people who work on this stuff.
> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.
So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.
Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:
So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.
Sad news, I put a lot of hope in this project. At least we keep the base interpreter improvements they brought along, and we still have PyPy. CPython JIT is dead. Long live PyPy!
Surprising, even though (IIRC) the performance of the JIT was always fairly underwhelming.
What about the other major change to Python's implementation: free threading? Has it lived up to expectations? I assume it's not also in doubt, despite Guido recently saying "Personally I wish we’d stuck with the GIL"? [0]
Getting a JIT from underwhelming to actually good is always going to be a whole lot of iterative work to remove blocking parts of the existing implementation. In Ruby-land we have had a good JIT for a while now, but it took many release cycles and a ton of resources before it was able to outperform the non JITed version when using Rails. Don't expect quick fixes that make great strides and any investments in it needs to be persistent and apply over time.
It boggles my mind that it apparently is the case that such a popular language as Python isn't able to get the resources to do this.
It is more than that. CRuby have had multiple JIT attempt before YJIT landed. From what I can tell CPython didn't have that. And luckily enough we now have ZJIT replacing YJIT.
Out of the box it's comparable to the current JIT, but yes, major gains are only possible with at least type annotations. But that may be the only real way forward for a JIT implementation for CPython with the ability to gradually speed up slow parts of large programs.
Sad to read this. I had hoped that with the advance of AI capabilities, features like this could be prototyped, tested, implemented and shipped even faster. And I was really looking forward to see performance gains for my favourite language.
Where do we go from here? Microsoft did not continue to finance the Faster CPython project and the current speed gains are good, but not what we hoped.
A biased opinion as I dev in F#, but it strikes me that AI is going to play best with functionally pure languages (strongly typed, no side effects, whatever we call that these days)?
F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.
The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.
And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).
I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"
Mature projects simply don't like large unexpected code drops if it is not made clear beforehand what the blast radius is and how it is going to be maintained. On top of that it seems that the performance gains weren't as expected and require further invasive changes.
Implementation effort has never been what has held these efforts back. LLM helps with squirting out code, but the hard tasks are benchmarking, profiling, community engagement, knowledge management, and dealing with the hard questions such as whether the change is actually worth the additional complexity.
These concerns won't go away even if we give up on human developers being able to grok the code and instead completely relying on LLMs for coding.
But I want to continue to understand the code. I dont know Rust (maybe AI can teach me?) and if I let Claude rewrite my current 50k LOC project in Rust, how can I understand what is happening and how can I work on it and think it further?
There are already many solutions to the performance issue from which you can pick. You don't need to understand the C in numpy to use it for example. And I'd say it's highly unlikely that you have a use case that, if it's applicable, isn't already addressed. Like it's generally pointless to optimize language performance for code that's doing a lot of network operations, because network lag will always dwarf such optimizations.
Rust is not your only option. Nor is it automatically the best option, though it is certainly a solid option for a certain set of requirements. Go is a pretty decent match for a Python rewrite; superficially it seems like you're giving up a lot but under the hood they're philosophically more compatible than meets the eye in a lot of ways. But any static language that you have some other local reasons to prefer is going to blow its socks off in terms of performance. And "blow its socks off" is just for the naive translation of a conventional Python code base over to a new language. If your codebase is amenable to threading and can take advantage of multiple CPUs somehow, the result may blow Python's socks off twice over.
The problem is that Python just isn't going to go fast. There are some languages that may have had more resources poured into them to make them go faster in absolute terms, like perhaps Java. But I don't think there's a single language that has had so many ways of trying to make it go faster given so much effort over so many decades. It's really time to call it... if you have Python code and it's a performance problem, if one of the many many many different solutions aren't adequate for some reason, Python is not going to get any faster than it is today and you're going to have to move off of it.
If you need high performance code, Python is simply not the right answer and you're going to have to skill up. Or accept what you have, which you'll have to decide if that's a viable option. If it's fast enough I don't advocate switching casually. But if it isn't, there isn't a realistic prospect that you're going to wake up in a year or two to a 30x-faster Python. I wasn't terribly convinced by that claim back in the early 2000s, but it hadn't really been tested yet. It has been tested now.
> RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
It's already typical to call into a native library with Python bindings for performance critical work in Python. Even Pydantic's validation runs in Rust code.
This works well if there are obvious hot spots that can be easily isolated, but sometimes the perf issues are more uniformly distributed throughout a codebase.
We pick languages that embrace machine code like Julia, Mojo, Common Lisp, Scheme, and stop calling libraries that are written in C, C++, Fortran, Rust, "Python Libraries", when they are thin bindings.
Clearly the community doesn't want PyPy, GraalPy, or even a CPython JIT.
> We want the PEP to be clear that it’s ultimately the JIT team’s responsibility to find solutions for these packages and tools, and how to deal with situations where a tool can’t be supported or made to work
Is it just me or is this absurdly impossible, to the point of willfully killing the project? Committing a team of brilliant compiler engineers to doing free consulting ad infinitum for every bit of tooling in the world, however hacked-together?
The alternative would be to have the bravery to say: this is the way forward to keep this language relevant into the future, and as a steering committee we collectively own that some tooling may be out of date. Instead, this is a signal that Python is going to rest on its laurels for a very long time.
Python use is widely diverse. Anyone wishing to contribute to that which is published to such a diverse community, all of whom expect Python to continue working for them without any regressions, has to be ready to cater to that diversity in its entirety.
Given that the speedup is modest (~10%) and the Python 3 migration caused enough pain for users, I get it though. If it was a 2x speedup it would be a different story IMO
I would not like a JIT to be embedded in python for one reason. The sole reason why Python didn't have all the security related bugs of other languages (just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities) is sorely because it's an interpreted language, that is python is compiled in a bytecode that runs in a VM.
Under these premises is very difficult that something not implemented correctly is really exploitable, and moreover it's simple to verify the security of the whole thing.
Opening a JIT, that would mean mapping memory pages as RWX, would open the door to all soft of exploits that now are not really possible.
While I don't disagree with you in principle, the context for JS JIT vulns and potential Python JIT vulns is pretty different and worth taking into consideration.
For JS, your machine running untrusted JS code is the intended use case. In this scenario the JS is sandboxed in whatever JS engine (V8 etc) and does not have access to "dangerous" methods, and the main use case for a JIT vuln is to escape this sandbox to execute code on the host. This applies to regular internet browsing (and visiting potentially malicious sites) and also, e.g., using the built-in browser in a locked down device (such as game consoles) to gain arbitrary code execution.
For Python, this security boundary doesn't exist. I can't really think of a scenario where you would be running potentially malicious Python code in CPython that would need a JIT vuln to escape whatever sandbox it is in instead of, say, calling the functions in Python directly (or using the C FFI to do whatever). I'm sure that in some niche use cases, this scenario might exist, but to me, it seems like the potential attack surface is fairly minimal.
EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT writes code to a writable mapping and then mprotects it to W^X? I don't think it leaves RWX pages lying around at all.
I would want a python implementation with a JIT and the ability to choose JIT or no-JIT when defining a python environment. And I want all my libraries to work either way.
I'll get what I want eventually but I might be retired by then.
Being slower than a compiled language is not saying much for an interpreted language. But it's slower than PHP which is a closer comparison, and PHP has had JIT for six years now.
Ruby has the reputation of being a very slow language, but Python has been slower still since Ruby jumped ahead with version 1.9, almost 20 years ago. Ruby also has had far more success with JIT compilers than Python has.
>Ruby also has had far more success with JIT compilers than Python has.
Ruby is on its third (I think?) JIT now? I'm a little surprised, given Ruby's reputation as a wacky "anything goes" language with a very complex grammar, and it makes me wonder why that is? Does it have to do with the design of the runtime itself more than the language semantics? Or is it more incidental, e.g. many popular ibs are C wrappers, already generates bytecode files, and so on.
Getting a JIT right is hard work because time spent by the JIT is time not spent doing actual work, and finding the correct tradeoffs is very much an empirical problem.
And PHP is quite the hot mess. Always has been, really. See the security issues over the years, including the most recent for example. Meanwhile I've never seen any serious issues for Python that's triggered mass required fixing. There's something to be said for moving carefully and ensuring things aren't broken to the best of one's ability, which is the SC's purpose.
Why had Python to succeed to Lisp in the AI wave? (rhetoric question)
Since PyPy and GraalPy will never get the love they deserve, I guess there is hope that Julia, Mojo, and the whole Rewrite in Go/Rust keep pushing along then.
I don't know why they don't just adopt Pypy as a faster Python implementation. Just let CPython develop slowly as the reference and give support to Pypy. It is already much faster than CPython.
The Rust based Python3 port can keep it and it can still be far more secure than CPython. Not everything it's JIT'ted on RustPython, it's ondemand per function, you have to enable it explicitly.
Interesting, makes sense. JIT in CPython main is a big deal, glad they’re proceeding with caution. Overall looks like they added it as an experimental feature for two years, then decided to back it out. Perfectly reasonable outcome.
It looks like it very much should never have happened like this. The actually hard and awkward questions have not been asked beforehand, and the SC doesn't like their hands being forced, and rightly so.
Assuming that the biggest bang-for-buck of JIT compilation is optimizing hot loops (which is also the biggest bang-for-buck of optimization in every other context as well), and assuming that, per the historical best practices, all hot loops in Python are already written in C (or Rust, these days), this outcome is somewhat what I would expect.
The performance of python indicates otherwise. There are some performance critical libraries written in C or C++ but a lot of python is written in python.
Python in itself is poorly suited to ever become performant as it lacks the types required. I imagine that causes some issues for effective jit-ing too.
They had a chance to fix this when adding type annotations, but instead of making it "optional/gradual typing" they opted to make it "documentation or linting support", which is obviously the wrong choice, but there were cultural reasons
Lol, it seems then they want to have a crippled ecosystem on purpose. I actually expected something good to come out of this, my mistake for trusting Python "engineers". Never again.
Python is my favorite language, but I'm honestly tired of having to rewrite performance bottlenecks in Cython or Rust all the time. When the Faster CPython project started with Microsoft's backing, I was really excited. Mark Shannon had that plan promising huge speedups across a few cycles, and while that clearly turned out to be much harder than anticipated, the team still kept pushing forward to deliver a real JIT.
Seeing all of that get bogged down in committee bureaucracy now is just frustrating.
I have a lot of respect for the core devs and I know maintaining CPython, and it is a time donation job. It isn't easy. But I feel that over-caution and red tape ends up discouraging serious contributions. They are setting an unreasonably high bar for something that would obviously help the entire ecosystem.
Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Python really deserves a proper JIT. I genuinely hope they reconsider this.
Unladen Swallow died for lack of upstream interest.
PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of versions behind CPython.
Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)
> Unladen Swallow died for lack of upstream interest
IIRC Unladen Swallow was, despite the extreme hype, nothing more than the internship project of a couple of students. It died because their internship was over.
Otherwise you're right, though. Lots of CPython alternatives suffer from lack of upstream interest - the Python language designers don't seem very interested in anything other than CPython. In particular, they appear to only consider CPython when adding features to the language. Two examples:
- Requiring `dict`s to be ordered because that's how they work in CPython, without considering how they work in other implementations
- Adding pattern matching syntax that's suitable for CPython's PEG parser, even though it's not LL(1) so can't be parsed by, say, MicroPython
> Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Every single time I remember the first AI wave was done with Lisp and Prolog, both with tooling that supports machine code generation since about 50 years.
That's what's kept the language and ecosystem improving and from becoming a hot mess. If something isn't properly specced then it really shouldn't be in prod. They allowed the experiment in the first place so it'd be easier to gather sufficient data to guide said speccing, but the ball was dropped by the proposers and so the experiment was an ultimate failure (unlike free threading) and prod is now being appropriately cleaned up.
It's not red tape, it's about making sure that the effort actually delivers what it promises and can continue to so in the future. From the state of the draft PEP it seems it doesn't and it won't ever. The SC merely put the nail into the coffin.
I know not everyone loves them; but `Claude rewrite the performance bottlenecks in Rust` works so well it is hard to see the security surface area and harder reason about performance as something that should be paid in core.
It's strange that typescript has 5 different LLM written competing static translators to compiled languages and python is killing JIT because the PEP was not accepted.
Unfortunate. I've been following the work on it occasionally.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
I think this is a major issue with requiring a PEP for what's a performance optimization.
JITs are big nasty things that by their nature have to do a lot of things. You are basically saying "Give us a PEP for a new python interpreter and compiler".
What they do is fuzzy by their nature because optimizations are very rarely universally beneficial. It's a game of measuring, looking for regressions, and tackling the biggest problem next for whatever sample applications you have available. You always run into the risk of overfitting on top of all that (You may have made django faster, but you've made flask slower).
This is the sort of thing that, IMO, should not go through the PEP process. It's an implementation detail for the runtime.
Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements. They'd never get done. Or if any javascript engine needed ECMA acceptance before they could do a new JIT optimization.
> Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements.
They do:
https://openjdk.org/jeps/197
https://openjdk.org/jeps/515
Not every optimization requires a JEP, but certainly something as big as "add or remove a JIT" would 100% require a JEP.
> Imagine, for example, if the JVM needed a bunch of JEPs for JIT improvements.
FWIW, I definitely can find a bunch of both PEPs and JEPs for internal optimizations, random examples:
https://peps.python.org/pep-0509/
https://openjdk.org/jeps/147
You can find those JEPs, but frankly they are there for 1 of 2 reasons.
1. An outsider to the regular OpenJDK dev team proposing an optimization.
2. Advertising for a cool new performance tweak with potentially large impact.
You'll also notice that the JEP descriptions are very different from PEP descriptions.
Probably the largest JEP for the JVM, Valhalla
https://openjdk.org/jeps/401
Does not have the same level of requirements being asked of the Python JIT. Further, it's well understood that this JEP will "live on main" for quite some time in preview before it's accepted and merged.
What the JEP does, instead, is simply describe the feature and potential risks.
The vast majority of JVM optimizations do not get a JEP.
Fair, I don’t argue with that, I’m not knowledgeable about Java’s processes. Just one thing:
> The vast majority of JVM optimizations do not get a JEP.
In Python too. Still, the JIT really is just a really big thing. I don’t know if there could be a better way to do it, as either 1. You develop it off-branch and… make an enormous PR when you’re done, with risks of getting rejected anyway? Or 2, you allow more experiments on main like Rust, but then you end up with hundreds of minor half-maintained nightly-only features, possibly conflicting with each other, and eventually go through bureaucracy to remove them anyway.
I mean, I have had great success with prototyping stuff in python, then letting an llm translate it to C. My 3090 with Gemma4:31b does this very well with a few guidelines.
Dunno how much investment is needed into language optimization anymore.
Here’s an idea instead (maybe it already exists). Allow a decorator for writing inline Cython. It solves, I think, 90% of the problem. On first run, or eagerly, compile the decorated functions. Basically, Cython is the JIT, and user enables it explicitly.
This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
Such a decorator would give you a low friction way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics, in particular "development semantics".
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
The main reason I don't see it in production code is you significantly increase your chance to write seg faults that kills the runtime / behaves unexpectedly to surrounding systems
This is fine for your own application code, but it doesn't do anything for the libraries you pull in for your project. I don't think that sprinkling Cython code in all library code is going to be a popular solution, especially for those not performance focused. The idea behind a JIT is to be a general performance solution not requiring changes in existing code.
I like Julia approach, compile everything on first launch, then reuse without speed penality, if you change a function, recompile that.
On another note, I'm working on a (big) PEP for python, and looking for maintainers and sponsors (as per python's rules). Do someone know where I could find such amazing persons? (I already have the code, works in cython,graalVM and pypy, and it's kind of a big change but I believe it will help a lot the language)
The language forum is the place to start- you should already have a thread there gathering feedback before writing a pep?
Thank-you, it's a longtime idea, but I was finally able to implement it, but now I'm trying to learn the procedure
There’s an #internals-and-peps channel in the Python discord where some folks hang out.
However because of this, Julia has has persistent problems with "time to first plot", which has gotten better, but they cannot fully ameliorate due to the way it fundamentally works. So there are tradeoffs -- the performance gets better, but the interpreted REPL experience is markedly worse.
This is largely the idea behind py2many + fquery, but we want to get rid of C-API as well.
I built this in 2001 (https://pyinline.sourceforge.net/). Obviously, that ancient thing doesn't work anymore, but the idea worked then and it ought to work today. That being said, I think there are significantly superior options in existence now!
This is numba?
Yeah, this is numba. I find it really frustrating the extent to which people don't know about it.
Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.
With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).
[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.
[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.
It's unfortunate that a technically exciting new feature is being killed partly because of what is essentially bureaucracy (and the lack of the developer's personal bandwidth). On the other hand, it's pretty safe to say that the strict adherence to the PEP policy and all of the bureaucracy that comes with it is what makes Python a great language, for better or worse.
Not to say that the technical objections were without merit, they likely very much weren't, and the debugger and instrumentation support and maintenance work sounds fairly painful on its own.
I think it would be a mistake to conclude this has anything at all to do with bureaucracy as a matter of inanimate policy that the SC was helplessly in thrall to. Bureaucracy was the tool the SC used to demonstrate that this feature lacked many important properties that they cared about, as mentioned: "maintenance, compatibility, and ... clear success metrics". For whatever detailed reasons, the PEP was withdrawn after it couldn't convincingly demonstrate those things to the satisfaction of the critics on the SC.
Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.
(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)
The problem is that clearly documenting all details as required in the PEP is at odds with complicated new features like a JIT where all the details can’t be known and have to be decided on as they are realized. Process / bureaucracy is at direct odds with experimentation.
But they weren’t asking for all the implementation details. They were asking for a clearly stated scope, success criteria, support plan, etc.
As they themselves pointed out, the problem isn’t that there’s anything wrong with an open-ended experiment. It’s just that open-ended experimentation doesn’t belong in main, of all places. They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
> clearly stated scope
The scope of a JIT is the entire python runtime. Not at the beginning, it can be more narrow, but certainly it can evolve to cover everything. JITs touch everything.
> success criteria
Impossible to measure. JITs don't universally make things faster and unless the SC was willing to accept "some programs will see around x% improvement" then it would be impossible to actually give them the criteria they want. There's a reason C++ compilers and their optimizations aren't defined by the ISO committee.
> support plan
Like most JITs, support for a JIT is an ongoing and ever evolving process. What's needed depends entirely on what's needed today. You usually need more than 1 or 2 dev working on it, which may be the issue here.
> It’s just that open-ended experimentation doesn’t belong in main, of all places.
Very strong disagree. That's simply the nature of performance optimizations for JITs. They belong on main because new optimizations and heuristics will always be generated. The role of doing a compiler is a long term experiment in performance, seeing what your runtime is currently doing and how it's being used and tweaking it to work better.
pypy isn't in a worse place because it's been running a longterm performance experiment.
> They have an ongoing duty to ensure that a critical piece of global infrastructure remains stable and trustworthy to consider.
None of the things mentioned actually do that, other than perhaps a support plan. There are plenty of critical infrastructure projects who all can manage to develop JITs. Python is the special snowflake that has decided to do JIT development incorrectly.
This is a runtime optimization/implementation detail. It's foolish to send it through a committee approval process. That's why you won't be able to find a single language with a JIT or advanced optimizations that has do it by committee.
And yet, not introducing unnecessary chaos into the CPython project remains important.
Perhaps these priorities were always irreconcilable.
Perhaps that’s why the existence of competing Python JIT projects that weren’t trying to commit directly into CPython’s main branch kept coming up in the conversation.
In that case I can understand why it's not a fit for Python. The risks you're describing are more what a brand new language would face: universal scope, impossible to measure success, no support plan... sounds like a nightmare in a mature product. If that's actually what it means to add a JIT to Python, the SC is doing its job by steering Python around this quixotic task.
> This is a runtime optimization/implementation detail.
I view JITs as a critical architectural component. Like you said, it impacts the whole system. Optimization and implementation details don't usually do that.
The problem is that “Python” the language includes native poorly isolated FFI interfaces exposed by CPython. So the language isn’t “Python” but CPython. That’s why alternate implementations like PyPy haven’t seen adoption despite having a very mature JIT while languages like JavaScript have JITs that put CPython’s performance to shame.
What are you basing this statement on?
The experiments had long been done. The topic at hand was including it as a core feature.
> Im not sure I even like calling it bureaucracy. It’s a process, yes. But we’re not talking many layers of abstract organizational hell, here. Were talking a fairly direct process of documenting what you are doing and how you are going to do it well enough that others working and depending on the project can understand it, coordinate with it, and have a reasonable degree of confidence that it won’t cause undue problems.
Bureaucracy doesn't definitionally mean Kafkaesque. It's popular to hate on it, but good or neutral bureaucracy does exist.
It doesn’t have to mean kafkaesque, but it does mean governance through layered hierarchies and fixed procedures.
In this case a single committee directly engaged with both core developers and the community, through a fairly open process, explicitly bent several of their own official rules to suit the occasion, and indicated cases where further flexibility might be needed depending on how things develop.
I would call that a pretty stellar example of non-bureaucratic governance.
HN when AI is used to bomb a school: technology is neutral.
HN when an organization achieves its intended goals: this can't be bureaucracy because it's good.
Looking at amount of JIT related (for example type confusion) security bugs in V8 I am not surprised other developers are vary of its inclusion.
Large codebases are bureaucracies of their own, that is, the complexity of extra layers starts to bog things down at some point.
In this case, it seems like the human bureaucracy made wise and measured decisions to contain the software bureaucracy. They made reasonable requests to clarify the scope and impact of a major, major core feature. The proposers didn't feel like that was doable, so a complexity nuke was avoided.
If you look at the reasons for this it seems like a good decision: https://discuss.python.org/t/pep-836-jit-go-brrr-the-path-to...
There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
>There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
And all of them are lacking in different ways, we ALSO, want faster pure Python.
They all have drawbacks, but they also reduce the case for needing to speed up CPython.
Pypy does provide faster pure Python. Its a compatible implementation with JIT.
> we don't feel we have the personal bandwidth to carry the current version forward
Are the JIT developers the same people who were working on Faster CPython at Microsoft? No surprise that they don't have the same amount of energy to dedicate to the JIT now that it's gone from a full-time job to a hobby project.
That team was mostly hired away by NVIDIA.
What's the status of 'Faster CPython'?
I wouldn't be surprised if they all went nodejs instead :/
While JIT is being dropped, the original proposer and leader of the initiative was hired by ARM and continues work on speeding up Python.
His current focus seems to be speeding up GC and free threading.
Microsoft is not financing it any more and laid off most of the team working on it.
Gonna dump a technical hot take here.
Python is a mature production language with a lot of users. Even before it was huge - like 20 years ago when I was in college - getting a small backwards compatible API change into a python extension module (gnu readline in my case) required convincing GvR personally that the API change was necessary.
Every JIT attempt I've seen for python just tries to cut too deep in the first pass, and scope themselves too large.
I believe any successful attempt at optimizing python performance from JIT will FIRST start with a 100% semantically compatible hidden type model for optimizing object property access and method dispatch.
The single biggest performance delta in most dynamic language jits come from that, not from optimizing guards off of bytecode operations or from compiling to machine code. It's from converting hashtable lookups into constant-time guard + offset lookups.
The hidden type model is also a hard nut to crack. It needs to perserve dict semantics which have very dynamic behaviour via the various `__*__` methods, and handle exotics scope objects since scoped name lookups are the most common ones which have their own exotic behaviours (I remember python providing some ability for callee frames to inspect caller scopes in some limited ways).
> It's from converting hashtable lookups into constant-time guard + offset lookups.
Python already does it in a lot of cases, see opcode specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.
I haven't kept up with python development lately, so thanks for pointing that out. I looked this up and it does seem like some sort of property access is being optimized. What's not clear is what the hidden runtime type model is being used for this, or if it's a lightweight tracking that captures some subset of property accesses.
More digging led to this: https://peps.python.org/pep-0659/ which seems to be what the effort is organized around?
Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.
The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
> Optimizing for this would require a tree of hidden map objects much like JS JIT engines use.
I don’t really think that’s true or even that relevant , at least in context of classes (especially with slots - the class, by definition, _is_ the “hidden map”) and not generic dicts.
> The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement.
Pretty sure the motivation was exactly the opposite - that a simple “if a LOAD_ATTR encounters the same class with slots N times in a row, swap it out for LOAD_ATTR_SLOT with inline cached slot index, and swap back to generic one if something changes” is way easier (both to implement and reason about) than any alternative that’d interact with the object model. (Also it means you skip having to branch between a slow and fast path at all)
> you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Sorry, that sounds a bit vague and not too different from what CPython already is doing.
Ah, the `__slots__` feature makes it make a lot more sense.
> Sorry, that sounds a bit vague and not too different from what CPython already is doing.
With slots being available, I can see where I was misled. I went looking for a type structure that modeled the dynamic properties on an object which would be used as a typeguard, but the class itself functions as that if you're restricting the optimization to slotted properties.
Appreciate the clarfication!
(Note that this optimization can also trigger for plain classes (that’s LOAD_ATTR_INSTANCE_VALUE), it just needs a bit more guards and is only useful if you don’t abuse instances being dynamic objects. I just used slots as the most obvious example, but there are lots of different op specializations since like 3.11)
It still blows my mind that the big "we're breaking backwards compatibility Python 3000" release didn't do anything to lay the groundwork for this, or any of Python's other structural problems.
For a long time Guido wanted CPython to remain simple, as the reference implementation.
Probably also during the 2 to 3 migration, but that goal appears to have now been abandoned.
I wonder how the authors of other implementations like pypy will be able to keep up as the reference implementation becomes more and more complex.
> the reference implementation.
The problem was he exposed too much of the implementation into the language itself, destroying its ability to be a simple reference implementation that others could re-implement.
This is slowly being addressed now, each version of Python sees the c API become more opaque.
At the time, Python had a clear purpose as an easy to learn "glue" language between other, perhaps more performant languages. There was no reason to make Python as fast because that would muddy its purpose by introducing compromises and complexity. For better or worse Python sat pretty well in its niche.
Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.
Its possible this could have been the concrete pillar that broke the camel's back and set Python 3 on the path of Perl 6.
A sad day for CPython but it doesn't have to be a sad day for Python. There is a fully maintained implementation of Python 3.13 that has an advanced JIT compiler. Once your Python warms up it can be 4x faster or higher (depending on what your program is doing). That implementation is called GraalPy and it builds on the JVM:
https://graalpy.org/
A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.
It has other interesting features
• Scalable pauseless GC
• Single binary distribution of Python programs (like Go)
• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.
• Supports native extensions. Tested and works with common libs like SciPy, NumPy, PyTorch, Pandas etc: https://graalpy.org/python-developers/compatibility/
• They're working on a GIL-free mode
In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.
https://www.youtube.com/watch?v=zo6L7bxn4OY
(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)
So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).
This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.
Disclosure: I sit next to some of the people who work on this stuff.
> Once your Python warms up
How long does that take?
The Pyronaut performance page has some graphs with the warmup parameters specified precisely:
https://pyronaut.io/performance/
> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.
So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.
Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:
https://medium.com/graalvm/escaping-the-giant-switch-dec20b5...
So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.
Sad news, I put a lot of hope in this project. At least we keep the base interpreter improvements they brought along, and we still have PyPy. CPython JIT is dead. Long live PyPy!
Surprising, even though (IIRC) the performance of the JIT was always fairly underwhelming.
What about the other major change to Python's implementation: free threading? Has it lived up to expectations? I assume it's not also in doubt, despite Guido recently saying "Personally I wish we’d stuck with the GIL"? [0]
[0] https://discuss.python.org/t/pep-805-safe-parallel-python/10...
Getting a JIT from underwhelming to actually good is always going to be a whole lot of iterative work to remove blocking parts of the existing implementation. In Ruby-land we have had a good JIT for a while now, but it took many release cycles and a ton of resources before it was able to outperform the non JITed version when using Rails. Don't expect quick fixes that make great strides and any investments in it needs to be persistent and apply over time.
It boggles my mind that it apparently is the case that such a popular language as Python isn't able to get the resources to do this.
It is more than that. CRuby have had multiple JIT attempt before YJIT landed. From what I can tell CPython didn't have that. And luckily enough we now have ZJIT replacing YJIT.
https://github.com/facebookincubator/cinderx might be a better approach.
CinderX isn't generally faster unless you use "Static Python" which is a major change in how programs are written compared to typical python code.
Out of the box it's comparable to the current JIT, but yes, major gains are only possible with at least type annotations. But that may be the only real way forward for a JIT implementation for CPython with the ability to gradually speed up slow parts of large programs.
https://facebookincubator.github.io/cinderx/#performance
Sad to read this. I had hoped that with the advance of AI capabilities, features like this could be prototyped, tested, implemented and shipped even faster. And I was really looking forward to see performance gains for my favourite language.
Where do we go from here? Microsoft did not continue to finance the Faster CPython project and the current speed gains are good, but not what we hoped.
A biased opinion as I dev in F#, but it strikes me that AI is going to play best with functionally pure languages (strongly typed, no side effects, whatever we call that these days)?
F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.
The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.
And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).
I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"
The biggest winner from AI indeed appears to be Rust.
Mature projects simply don't like large unexpected code drops if it is not made clear beforehand what the blast radius is and how it is going to be maintained. On top of that it seems that the performance gains weren't as expected and require further invasive changes.
Implementation effort has never been what has held these efforts back. LLM helps with squirting out code, but the hard tasks are benchmarking, profiling, community engagement, knowledge management, and dealing with the hard questions such as whether the change is actually worth the additional complexity.
These concerns won't go away even if we give up on human developers being able to grok the code and instead completely relying on LLMs for coding.
At this point if you have a python codebase where perf is an issue, the "AI-pilled" answer is to rewrite it in a faster language.
But I want to continue to understand the code. I dont know Rust (maybe AI can teach me?) and if I let Claude rewrite my current 50k LOC project in Rust, how can I understand what is happening and how can I work on it and think it further?
There are already many solutions to the performance issue from which you can pick. You don't need to understand the C in numpy to use it for example. And I'd say it's highly unlikely that you have a use case that, if it's applicable, isn't already addressed. Like it's generally pointless to optimize language performance for code that's doing a lot of network operations, because network lag will always dwarf such optimizations.
Rust is not your only option. Nor is it automatically the best option, though it is certainly a solid option for a certain set of requirements. Go is a pretty decent match for a Python rewrite; superficially it seems like you're giving up a lot but under the hood they're philosophically more compatible than meets the eye in a lot of ways. But any static language that you have some other local reasons to prefer is going to blow its socks off in terms of performance. And "blow its socks off" is just for the naive translation of a conventional Python code base over to a new language. If your codebase is amenable to threading and can take advantage of multiple CPUs somehow, the result may blow Python's socks off twice over.
The problem is that Python just isn't going to go fast. There are some languages that may have had more resources poured into them to make them go faster in absolute terms, like perhaps Java. But I don't think there's a single language that has had so many ways of trying to make it go faster given so much effort over so many decades. It's really time to call it... if you have Python code and it's a performance problem, if one of the many many many different solutions aren't adequate for some reason, Python is not going to get any faster than it is today and you're going to have to move off of it.
If you need high performance code, Python is simply not the right answer and you're going to have to skill up. Or accept what you have, which you'll have to decide if that's a viable option. If it's fast enough I don't advocate switching casually. But if it isn't, there isn't a realistic prospect that you're going to wake up in a year or two to a 30x-faster Python. I wasn't terribly convinced by that claim back in the early 2000s, but it hadn't really been tested yet. It has been tested now.
Pypy is really fast. Have you tried it?
RustPython it's pretty mature.
Not at all.
> RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
- https://github.com/RustPython/RustPython
You could try having it rewrite parts of the hot path into Rust with PyO3.
It's already typical to call into a native library with Python bindings for performance critical work in Python. Even Pydantic's validation runs in Rust code.
This works well if there are obvious hot spots that can be easily isolated, but sometimes the perf issues are more uniformly distributed throughout a codebase.
We pick languages that embrace machine code like Julia, Mojo, Common Lisp, Scheme, and stop calling libraries that are written in C, C++, Fortran, Rust, "Python Libraries", when they are thin bindings.
Clearly the community doesn't want PyPy, GraalPy, or even a CPython JIT.
From https://discuss.python.org/t/pep-836-jit-go-brrr-the-path-to... :
> We want the PEP to be clear that it’s ultimately the JIT team’s responsibility to find solutions for these packages and tools, and how to deal with situations where a tool can’t be supported or made to work
Is it just me or is this absurdly impossible, to the point of willfully killing the project? Committing a team of brilliant compiler engineers to doing free consulting ad infinitum for every bit of tooling in the world, however hacked-together?
The alternative would be to have the bravery to say: this is the way forward to keep this language relevant into the future, and as a steering committee we collectively own that some tooling may be out of date. Instead, this is a signal that Python is going to rest on its laurels for a very long time.
Python use is widely diverse. Anyone wishing to contribute to that which is published to such a diverse community, all of whom expect Python to continue working for them without any regressions, has to be ready to cater to that diversity in its entirety.
Given that the speedup is modest (~10%) and the Python 3 migration caused enough pain for users, I get it though. If it was a 2x speedup it would be a different story IMO
What’s the latest on getting rid of the GIL? Did that work out?
It did. It's slower though.
I would not like a JIT to be embedded in python for one reason. The sole reason why Python didn't have all the security related bugs of other languages (just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities) is sorely because it's an interpreted language, that is python is compiled in a bytecode that runs in a VM.
Under these premises is very difficult that something not implemented correctly is really exploitable, and moreover it's simple to verify the security of the whole thing.
Opening a JIT, that would mean mapping memory pages as RWX, would open the door to all soft of exploits that now are not really possible.
While I don't disagree with you in principle, the context for JS JIT vulns and potential Python JIT vulns is pretty different and worth taking into consideration.
For JS, your machine running untrusted JS code is the intended use case. In this scenario the JS is sandboxed in whatever JS engine (V8 etc) and does not have access to "dangerous" methods, and the main use case for a JIT vuln is to escape this sandbox to execute code on the host. This applies to regular internet browsing (and visiting potentially malicious sites) and also, e.g., using the built-in browser in a locked down device (such as game consoles) to gain arbitrary code execution.
For Python, this security boundary doesn't exist. I can't really think of a scenario where you would be running potentially malicious Python code in CPython that would need a JIT vuln to escape whatever sandbox it is in instead of, say, calling the functions in Python directly (or using the C FFI to do whatever). I'm sure that in some niche use cases, this scenario might exist, but to me, it seems like the potential attack surface is fairly minimal.
EDIT: Also, I'm pretty sure that Python's copy-and-patch JIT writes code to a writable mapping and then mprotects it to W^X? I don't think it leaves RWX pages lying around at all.
Who is running sandboxed untrusted Python code?
JS has a completely different threat model. And PHP has just been a basketcase from inception.
> just think about all the exploits in JS engines, as well as the recent PHP vulnerabilities
Can you name some of these JS engine vulnerabilities? And the PHP one had nothing to do with JIT.
Not OP, but JS engine JIT bugs are pretty common.
https://phrack.org/issues/70/exploiting-logic-bugs-in-javasc...
https://www.sentinelone.com/vulnerability-database/cve-2026-...
https://projectzero.google/2020/09/jitsploitation-one.html
https://www.zerodayinitiative.com/blog/2021/12/6/two-birds-w...
I would want a python implementation with a JIT and the ability to choose JIT or no-JIT when defining a python environment. And I want all my libraries to work either way.
I'll get what I want eventually but I might be retired by then.
What a shame. The alternative is CinderX? It's slower unless you stick to "Static Python" which is not most python code in the wild.
Hope the copy-and-patch work gets a proper write-up before it’s removed... The technique is interesting well beyond CPython.
I hope it does not lead to performance issues later. Python is already famous for being slower than compiled languages.
Being slower than a compiled language is not saying much for an interpreted language. But it's slower than PHP which is a closer comparison, and PHP has had JIT for six years now.
Ruby has the reputation of being a very slow language, but Python has been slower still since Ruby jumped ahead with version 1.9, almost 20 years ago. Ruby also has had far more success with JIT compilers than Python has.
>Ruby also has had far more success with JIT compilers than Python has.
Ruby is on its third (I think?) JIT now? I'm a little surprised, given Ruby's reputation as a wacky "anything goes" language with a very complex grammar, and it makes me wonder why that is? Does it have to do with the design of the runtime itself more than the language semantics? Or is it more incidental, e.g. many popular ibs are C wrappers, already generates bytecode files, and so on.
Getting a JIT right is hard work because time spent by the JIT is time not spent doing actual work, and finding the correct tradeoffs is very much an empirical problem.
And PHP is quite the hot mess. Always has been, really. See the security issues over the years, including the most recent for example. Meanwhile I've never seen any serious issues for Python that's triggered mass required fixing. There's something to be said for moving carefully and ensuring things aren't broken to the best of one's ability, which is the SC's purpose.
This is why we cannot have nice things.
Why had Python to succeed to Lisp in the AI wave? (rhetoric question)
Since PyPy and GraalPy will never get the love they deserve, I guess there is hope that Julia, Mojo, and the whole Rewrite in Go/Rust keep pushing along then.
I don't know why they don't just adopt Pypy as a faster Python implementation. Just let CPython develop slowly as the reference and give support to Pypy. It is already much faster than CPython.
The Rust based Python3 port can keep it and it can still be far more secure than CPython. Not everything it's JIT'ted on RustPython, it's ondemand per function, you have to enable it explicitly.
Interesting, makes sense. JIT in CPython main is a big deal, glad they’re proceeding with caution. Overall looks like they added it as an experimental feature for two years, then decided to back it out. Perfectly reasonable outcome.
It looks like it very much should never have happened like this. The actually hard and awkward questions have not been asked beforehand, and the SC doesn't like their hands being forced, and rightly so.
Assuming that the biggest bang-for-buck of JIT compilation is optimizing hot loops (which is also the biggest bang-for-buck of optimization in every other context as well), and assuming that, per the historical best practices, all hot loops in Python are already written in C (or Rust, these days), this outcome is somewhat what I would expect.
The performance of python indicates otherwise. There are some performance critical libraries written in C or C++ but a lot of python is written in python.
Python in itself is poorly suited to ever become performant as it lacks the types required. I imagine that causes some issues for effective jit-ing too.
They had a chance to fix this when adding type annotations, but instead of making it "optional/gradual typing" they opted to make it "documentation or linting support", which is obviously the wrong choice, but there were cultural reasons
Lol, it seems then they want to have a crippled ecosystem on purpose. I actually expected something good to come out of this, my mistake for trusting Python "engineers". Never again.
This is pretty bittersweet to see.
Python is my favorite language, but I'm honestly tired of having to rewrite performance bottlenecks in Cython or Rust all the time. When the Faster CPython project started with Microsoft's backing, I was really excited. Mark Shannon had that plan promising huge speedups across a few cycles, and while that clearly turned out to be much harder than anticipated, the team still kept pushing forward to deliver a real JIT.
Seeing all of that get bogged down in committee bureaucracy now is just frustrating.
I have a lot of respect for the core devs and I know maintaining CPython, and it is a time donation job. It isn't easy. But I feel that over-caution and red tape ends up discouraging serious contributions. They are setting an unreasonably high bar for something that would obviously help the entire ecosystem.
Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Python really deserves a proper JIT. I genuinely hope they reconsider this.
Unladen Swallow died for lack of upstream interest.
PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of versions behind CPython.
Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)
I’m not optimistic.
> Unladen Swallow died for lack of upstream interest
IIRC Unladen Swallow was, despite the extreme hype, nothing more than the internship project of a couple of students. It died because their internship was over.
Otherwise you're right, though. Lots of CPython alternatives suffer from lack of upstream interest - the Python language designers don't seem very interested in anything other than CPython. In particular, they appear to only consider CPython when adding features to the language. Two examples:
- Requiring `dict`s to be ordered because that's how they work in CPython, without considering how they work in other implementations
- Adding pattern matching syntax that's suitable for CPython's PEG parser, even though it's not LL(1) so can't be parsed by, say, MicroPython
> Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Every single time I remember the first AI wave was done with Lisp and Prolog, both with tooling that supports machine code generation since about 50 years.
> committee bureaucracy
That's what's kept the language and ecosystem improving and from becoming a hot mess. If something isn't properly specced then it really shouldn't be in prod. They allowed the experiment in the first place so it'd be easier to gather sufficient data to guide said speccing, but the ball was dropped by the proposers and so the experiment was an ultimate failure (unlike free threading) and prod is now being appropriately cleaned up.
It's not red tape, it's about making sure that the effort actually delivers what it promises and can continue to so in the future. From the state of the draft PEP it seems it doesn't and it won't ever. The SC merely put the nail into the coffin.
I know not everyone loves them; but `Claude rewrite the performance bottlenecks in Rust` works so well it is hard to see the security surface area and harder reason about performance as something that should be paid in core.
Done https://github.com/can1357/pon
Seems unmaintained from July.
It doesn’t matter. Just write it in Rust. AI will make Python less relevant in the future.
It seems like such a failing and waste of human life that we spend all this time waiting around for python when the JS engines are so cutting edge.
It's strange that typescript has 5 different LLM written competing static translators to compiled languages and python is killing JIT because the PEP was not accepted.
cPython is killing it. You are very free to vibe code a new python interpreter that does JIT.
In the remote eventuality that it should actually work and be maintained up to date and secure, I'm sure it will acquire users.
More interested in creating a variant of python that's practical.
Wrote a parser with a rust compatible pattern matching syntax here:
https://github.com/py2many/spy-ast
Should help both JIT and static translation.
I just use Go and Rust for everything; most of my mobile apps are in Flutter. I don't see any point in using anything else.
for me it's over for python