This is one of the (several?) things that make me very worried about Rust long-term. I love the language, and reach for it even when it sometimes isn't the most appropriate thing. But reading some of the made-up syntax in the "Removing Coherence" section makes my head hurt.
When I used to write Scala, I accepted the fact that I don't have a background in type/set/etc. theory, and that there were some facets of the language that I'd probably never understand, and some code that others had written that I'd probably never understand.
With a language like Rust, I feel like we're getting there. Certain GAT syntxes sometimes take some time for me to wrap my head around when I encounter them. Rust feels like it shouldn't be a language where you need to have some serious credentials to be able to understand all its features and syntax.
On the other end we have Go, which was explicitly designed to be easy to learn (and, unrelatedly, I don't like for quite a few reasons). But I was hoping that we could have a middle ground here, and that Rust could be a fully-graspable systems-level language.
Then again, for more comparison, I haven't used C++ since before they added lambdas. I wonder if C++ has some hairy concepts and syntax today on par with Rust's more difficult parts.
> I wonder if C++ has some hairy concepts and syntax today
Both better and worse.
The current version of idiomatic C++ is much cleaner, more concise, and more powerful than the version of C++ you are familiar with. You don't need C-style macros anymore. The insane template metaprogramming hacks are gone. Some important things that were problematic to express in C++ (and other systems languages to be fair) are now fully defined e.g. std::launder. C++ now has expansive compile-time programming features, which is killer for high-performance systems code, and is more expressive than Rust in important ways.
The bad news is that this was all piled on top of and in addition to the famous legacy C++ mess for backward compatibility. If you are mixing and matching ancient C++ with modern C++, you are going to have a bad time. That's the worst of all worlds.
But if you are lucky enough to work with e.g. an idiomatic C++20 code base, it is a much simpler and better language than legacy C++. I've done a few major C++ version upgrades to code bases over the years; the refactored code base was always smaller, cleaner, safer, and easier to maintain than the old version.
These problems will happen in some form for every long lived codebase. In 15 years you will have some regrets, and some of those regrets will be things that are all core APIs used everywhere in the code base and so impossible to quickly change.
Isn't it 100x worse in C++ though? Because the surface area is huge? I work on legacy apps but PHP 5.6 is not that fundamentally different from PHP 8.3.
It's much simpler and better than it used to be but it's still pretty bad. As just one example off the top of my head consider the meaning of curly braces for initialization. There's several different things they can mean depending on the context. Good luck figuring out which set is currently in effect.
The initialization situation in C++ is indefensibly broken. It is near the top of my list of things I hate about C++.
You can mitigate it with some practices but that this is even necessary is a crime. Initialization is one of the most basic things in software development. How do you fuck it up so badly?
On a day to day basis it doesn’t cause me issues but it offends me just on principle.
I still use Eclipse CDT and its static analysis is running in real time, as you type code, which is killer. Combined with Valgrind integration, I don't see myself moving on anytime soon.
Is Eclipse CDT still good these days? Wow did not hear of it for a while. I thought C++ support was not maintained anymore.
I use CLion mostly but I never stop coming back to Emacs+LSP.
And yes, the analysis is quite competitive tbh. People often talk about this weird thing or the other in C++ but the experience is quite better than what the ISO standard strictly has to offer.
Eclipse is getting stable releases four times a year (i.e. every three months). C/C++ support is also being actively maintained and is pretty fast these days.
Eclipse is one of the rare software suites which didn't get slower as the tech evolves. Yes, it's probably heavier when compared to 20 years ago, but it starts pretty quickly and works snappily. I'm a happy camper.
If only the Go tools didn't get discontinued, but alas. KATE/BBEdit + Gopls is a pretty nifty combo on Linux/macOS.
> I wonder if C++ has some hairy concepts and syntax today on par with Rust's more difficult parts.
… … … … Unqualified name lookup has been challenging in C++ since even before C++11. Overload resolution rules are so painful that it took me weeks to review a patch simply because I had to back out of trying to make sense of the rules in the standard. There's several slightly different definitions of initialization. If you really want to get in the weeds, starting playing around with std::launder and std::byte and strict aliasing rules and lifetime rules, and you'll yearn for the simplicity of Rust.
C++ is the absolute most complex of any of the languages whose specifications I have read, and that's before we get into the categories of things that the standard just gives up on.
> starting playing around with std::launder and std::byte and strict aliasing rules and lifetime rules, and you'll yearn for the simplicity of Rust
Annotations like std::launder, lifetime manipulation, etc solve a class of problems that exist in every systems language. They inform the compiler of properties that cannot be known by analyzing the code. Rust isn't special in this regard, it has the same issues.
Without these features, we either relied on unofficial compiler-specific behavior or used unnecessarily conservative code that was safe but slower.
> Rust isn't special in this regard, it has the same issues.
This is both fundamentally true and misleading. Rust has to solve the same issues but isn't obliged to make all the same bad choices to do that and so the results are much better.
For example C++ dare not perform compile time transmutations so, it just forbids them and a whole bunch of extra stuff landed to work around that, but in Rust they're actually fine and so you can just:
That blows up at compile time because we claimed the bit pattern for the integer 2 is a valid boolean and it isn't. If we choose instead 0 (or 1) this works and we get the expected false (or true) boolean instead of a compiler diagnostic.
C++ could allow this but it doesn't, rather than figure out all the tricky edge cases they just said no, use this other new thing we made.
> For example C++ dare not perform compile time transmutations
I am confused by this assertion. You can abuse the hell out of transformations in a constexpr context. The gap between what is possible at compile-time and run-time became vanishingly small a while ago.
I think your example is not illustrative in any case. Many C++ code bases work exactly like your example, enforced at compile-time. That this can be an issue is a hangover from retaining compatibility with C-style code which conflates comparison operators and cast operators. It is a choice.
C++ can enforce many type constraints beyond this at compile-time that Rust cannot, with zero effort or explicit type creation. No one should be passing ints around.
Surely "We have many different ways to do this, each with different rules" is exactly the point? C++ 20's std::bit_cast isn't necessarily constexpr by the way although it is for the trivial byte <-> boolean transmutation I mentioned here.
I see that C++ people were more comfortable with the "We have far too many ways to initialize things" examples of this problem but I think transmutation hits harder precisely because it sneaks up on you.
bit_cast and reinterpret_cast do different things: one works at the value level, the second preserves address identity (and it is problematic from an aliasing point of view).
Not sure what any of this has to do with initialization though.
FWIW, the direct translation of your rust code is:
constexpr char y = 2;
constexpr bool x = std::bit_cast<bool>(y);
It fails on clang for y=2 and works for y=1, exactly like rust;
GCC produces UB for y=2, I don't know if it is a GCC bug or the standard actually allows this form of UB to be ignored at contexpr time.
What is the rust equivalent of reinterpret_cast and does it work at constexpr time?
edit: I guess it would be an unsafe dereference of a casted pointer. Does it propagate constants?
Firstly, that's not a direct translation because you're making two variables and I made none at all. Rust's const is an actual constant, it's not an immutable variable. We have both, but they're different. The analogous Rust for your bit cast example would make two immutable variables that we promise have constant values, maybe:
Of course this also won't compile, because the representation for 2 still isn't a boolean. If it did compile you'd also (by default) get angry warnings because it's bad style to give these lowercase names.
I also don't know if you found a GCC bug but it seems likely from your description. I can't see a way to have UB, a runtime phenomenon, at compile time in C++ as the committee imagines their language. Of course "UB? In my lexer?" is an example of how the ISO document doesn't understand intention, but I'd be surprised if the committee would resolve a DR with "That's fine, UB at compile time is intentional".
I understand that "these are different things" followed by bafflegab is how C++ gets here but the whole point of this sub-thread is that Rust didn't do that, so in Rust these aren't "different things". They're both transmutation, they don't emit CPU instructions because they happen in the type system and the type system evaporates at runtime.
So this is an impedance mismatch, you've got Roman numerals and you can't see why metric units are a good idea, and I've got the positional notation and so it's obvious to me. I am not going to be able to explain why this is a good idea in your notation, the brilliance vanishes during translation.
I'm using two variables because numeric literals have the wrong type and bit_cast rejects transmutations between differently sized types.
I could have written it as x = bit_cast<bool>(char{2}), but does it really make a difference?
I don't know enough rust to know what's the difference between its const and c++ constexpr. It might not be a meaningful difference in C++.
> So this is an impedance mismatch, you've got Roman numerals and you can't see why metric units are a good idea, and I've got the positional notation and so it's obvious to me. I am not going to be able to explain why this is a good idea in your notation, the brilliance vanishes during translation.
There are plenty of rust users on HN that are capable of kind, constructive, and technically interesting conversations. Unfortunately there are a small few that will destroy any goodwill the rest of the community works hard to generate.
> I could have written it as x = bit_cast<bool>(char{2}), but does it really make a difference?
Not really, that's also a variable. We're running into concrete differences here, which is what I was gesturing at. In C++ you've got two different things, one old and one new, and the new one does some transmutations (and is usually constexpr) while the old one does others but isn't constexpr. It's not correct to say that reinterpret_cast isn't a transmutation, for example it's the recognised way to do the "I want either a pointer or an integer of the same size" trick in C++ which is exactly that. Let me briefly explain, as much to ensure it's clear in my head as yours:
In C++ we have an integer but sometimes we're hiding a pointer in there using reinterpret_cast, in Rust we have a pointer but sometimes we're hiding an integer in there using transmute [actually core::ptr::without_provenance but that's just a transmute with a safe API]. Of course the machine code emitted is identical, because types evaporate at compile time the CPU doesn't care whether this value in a register "is" a pointer or not.
Anyway, yes the issues are the same because ultimately the machines are the same, but it's not true that C++ solved these issues the only way they could be addressed, better is possible. And in fact it would surely be a disappointment if we couldn't do any better decades later. I hope that in twenty years the Rust successor is as much better.
I don't know a way to express actual constants in C++ either. If there isn't one yet maybe C++ 29 can introduce a stuttering type qualifier co_co_const to signify that they really mean constant this time. Because constexpr is a way to get an immutable variable (with guaranteed compile time initialization and some other constraints) and in C++ we're allowed to "cast away" the immutability, we can actually just modify that variable, something like this: https://cpp.godbolt.org/z/EYnWET8sT
In contrast it doesn't mean anything to modify a constant in either language, it's not a surprise that 5 += 2 doesn't compile and so likewise Rust's core::f32::consts::PI *= 2; won't compile, and if we made our own constants we can't change those either. We can write expressions where we call into existence a temporary with our constant value, and then we mutate the temporary, but the constant itself is of course unaffected if we do this.
This can be a perf footgun, you will see newcomers write Rust where they've got a huge constant (e.g a table of 1000 32-bit floating point numbers) and they write code which just indexes into the constant in various parts of their program, if the index values are known at compile time this just optimises to the relevant 32-bit floating point number, because duh, but if they aren't it's going to shove that entire table on your stack everywhere you do this, and that's almost certainly not what you intended. It's similar to how newcomers might accidentally incur copies they didn't mean in C++ because they forgot a reference.
I'm afraid that just C++ being C++ and you are deep into UB; you can't really modify a constexpr value at runtime, and if you cast away its constness with what is effectively a const cast you are on your own. This will print "0 3" which is obviously nonsense:
constexpr int x = 3;
((int&)x) = 0;
char y[x];
std::print("{}, {}",x, sizeof(y));
The output might change according to the compiler and optimization level.
You can also move the problematic sequence into a constexpr function and invoke it in a constexpr context: the compiler will reject the cast now.
Enum constants can't become lvalues, so the following also won't compile:
enum : int { x = 3};
((int&)x) = 0;
So I guess that's closer to the rust meaning of constant.
FWIW, notoriously you could modify numeric literals in early Fortrans and get into similar nonsense UB.
edit: in the end[1] it seems that you take exception with constexpr not being prvalues in C++. I guess it was found more convenient for them to have an address [1]. That doesn't make them less constant.
[1] or at least I think you do, it is not clear to me what you have been trying to claim in this discussion.
[2] C++ will materialize prvalues into temporaries when their address is needed (and give them an unique address), I guess it was thought to be wasteful for large constexpr objects, and avoids the rust pitfall you mentioned.
> I'm afraid that just C++ being C++ and you are deep into UB
Of course, but the reason to even do this is only to illustrate that it's just another variable, nothing more. As much for myself as for you.
I've heard the stories about older languages but I assume that's not a thing on a modern Fortran and I'm sure we agree it's a bad idea.
The main thrust of this sub-thread was that languages can, and I believe Rust did, choose to solve the same issues but in a better way and so "This is too complicated in C++" doesn't translate to "It will be too complicated in every language". I think some C++ people have a version of the "End of History" nonsense, which is a nineteenth century idea. If you think noteworthy change happened after that and so history didn't end then hopefully you agree that makes no sense for general world history, and perhaps you can agree likewise C++ isn't the final apex of programming languages.
Is there a reason Rust would not (as it was done in the ‘good ole days’) index the table via pointer arithmetic from .data? Also, I’m assuming that because you are discussing new devs, that they are not making the implementation decision to place the table on the heap and using Rist’s subscript operator, which I would understand Rust not doing as default. I can not think of a reason that the table should ever be put on the stack for reading a single value, so that being the default seems an oddly pessimistic default. I could be missing something regarding how Rust handles literal data ‘written out’ into source though.
The table is on the stack because we conjured into existence a temporary, so it's exactly as if the programmer had conjured the variable as a local by hand. Suppose our table is named SOUP
const SOUP: [f32; 1000] = [ /* whatever */ ];
let foo = something(blah_blah, blah) * SOUP[4];
// The optimiser will see that SOUP[4] is exactly say 1.5_f32 so it'll just do
// the same as if we'd calculated something(blah_blah, blah) * 1.5_f32
However
let foo = something(blah_blah, blah) * SOUP[blah];
Now blah is a variable, we might need any value from SOUP, the optimiser doesn't know what values it might have - so a temporary is conjured into existence, equivalent to:
let tmp = SOUP;
let foo = something(blah_blah, blah) * tmp[blah];
You do presumably recognise that this now puts SOUP on the stack right? The temporary is equivalent, but without the explicitness.
Now if you know what you're doing you would of course have one single place in your program where you do this:
static SOUP: [f32; 1000] = [ /* whatever */ ];
And now there's an immutable global named SOUP and like "the good ole days" we don't keep writing this huge data blob to the stack only to subsequently look at a single element. But that's not the thing the noob wrote so that's not what they get.
"Sufficiently smart compilers" are a very gradual miracle. In theory a compiler could realise this is a good idea, in practice today I doubt you will find such a compiler, so just write explicitly that you want a single variable if that's what you want.
In my experience conversions is one of the things that maximum warning levels do excellent static analysis for nowadays. In the last 15 years I hardly had a couole problems (init vs paren initialization). All narrowing etc. is caught out of the box with warnings.
My point is, in your exact example both reinterpret_cast and C-style casts have the exact same behavior, making the example bad. If you want to showcase a deficiency of C++, it would make sense to pick something where the difference between cast types actually matters.
The right strategy to use C++ efficiently is to set warnings to the maximum as errors and take the core guidelines or similar and avoid past cruft.
More often than not (except if you inherit codebases but clang has a modernize tool) most of the cruft is either avoidable or caught by analyzers. Not all.
But overall, I feel that C++ is still one of the most competitive languages if you use it as I said and with a sane build system and package manager.
> But reading some of the made-up syntax in the "Removing Coherence" section makes my head hurt.
Articles discussions new features always have difficult syntax. There have been proposals like this going on from the start.
Fortunately the language team is cognizant of the syntax and usability issues with proposals. There have been a lot of proposals that started off as very unwieldy syntax but were iterated for years until becoming more ergonomic.
I think it's more than the syntax, it's just the number of concepts you need to keep in your head to read a type signature, or a trait declaration, or whatever. There's only so much you can pack into a limited language before it becomes too much to keep in your head at once.
Most of the time over the past couple years when somebody complained about Rust getting a complex new feature, it was complex backend work that exhibited to users as the removal of restriction, making the language simpler. Example: async closures. A language that supports both async and closures but doesn't support them together is more complicated than one that supports async closures.
This feels like it should end up similar. The driving desire here is the removal of a restriction, so hopefully it ends up as an end user simplification rather than complication.
I really wouldn't worry much. Over the last decade of rust, very few of the articles exploring new syntax have turned into anything controversial by the time they were merged (I can't even think of big rust syntax changes other than `impl T` in arguments etc). The singular example really is async/await and, having been quite worried about it / against it at the time, it was really nothing to be concerned with at all.
Having used Rust professionally for six years now, I share your fear. Like many of the commenters below, coherence just hasn't been a big problem for me. Maybe there are problem spaces where it's particularly painful?
How does the Rust language team weigh the benefits of solving user problems with new language features against the resulting increased complexity? When I learned Rust, I found it to be quite complex, but I also got real value from most of the complexity. But it keeps growing and I'm not always sure people working on the language consider the real cost to new and existing users when the set of "things you have to know to be competent in the language" grows.
You can look at the discussions in any of the language RFCs to see that increased complexity is one of the recurring themes that get brought up. RFCs themselves have a "how do we teach this?" section, that IMO makes or break a proposal.
Keep in mind that as time goes on, features being introduced will be more and more niche. If you could do things in a reasonable way without the new feature, the feature wouldn't be needed. That doesn't mean that everyone needs to learn about the feature, only the people that need that niche have to even know about it (as long as it is 1) it interacts reasonably with the rest of the language, 2) its syntax is reasonable in that it is either obvious what's going on or easy to google and memorable so that you don't have to look it up again and 3) it is uncommon enough that looking at a random library you won't be seeing it pop up).
Rust opened the door to innovation in the low-level languages space, but as long as it is already the most theoretically advanced practical language there, it will always attract the audience that actually wants to push it further. I don't know if there is a way to satisfy both audiences.
A similar thing happened with C++: The fact that it had a relatively high complexity of interacting features (already 30+ years ago) that you could use to do smart things, did attract smart people with a high tolerance for complexity. And with such an audience, complexity tends to expand up to the limits of tolerance (or even a little beyond).
Rust had a better start, not the least because it wasn’t designed on top of an existing language like C++ was, but who knows what it will look like in 30 years.
I think there is: a schism. Another language, inspired, intelligible and interoperable with Rust, but with other goals, likely ease of use or surface simplicity. In my mind it would be pretty much the same as Rust, but whenever a compile error gives you a suggestion in rustc would instead compile (and at most be a warning in this hypothetical language). Migrating from Rust to this language would be changing a single setting in Cargo.toml. The other way around would be fixing a bunch of compile errors. You could use the entire crate ecosystem in a native way. This language could also serve as a test bed for features that might or might not be suitable for Rust. It can also have a more aggressive evolution schedule, meaning that it wouldn't be perma-1.x, so it can be bolder on what is attempted.
There is precedent: with type checkers like pyright you can opt into specific checks, or have a basic, standard, strict setting, each expanding the set of checks done.
How would dependencies work in this schism? E.g. if serde starts using named impls, do all dependencies have to use named impls?
I mean… Sure, if we’re just making stuff up, a compiler that can magically understand whatever you were trying to do and then do that instead of what you wrote, I guess that’s a nice fantasy?
But out here on this miserable old Earth I happen to think that Rust’s errors are pretty great. They’re usually catching things I didn’t actually intend to do, rather than preventing me from doing those things.
> But out here on this miserable old Earth I happen to think that Rust’s errors are pretty great. They’re usually catching things I didn’t actually intend to do, rather than preventing me from doing those things.
As it happens, you are replying to the person who made Rust's errors great! (it wasn't just them of course, but they did a lot of it)
Yeah, for me reflection and coroutines were the first changes to C++ where the implementation and use mechanics weren't immediately obvious by reading a few references. It requires a bit of proper study to wrap your head around it.
I used Rust for ~14 months and released one profitable SaaS product built entirely in Rust (actix-web, sqlx, askama).
I won't be using Rust moving forward. I do like the language but it's complicated (hard to hold in your head). I feel useless without the LSP and I don't like how taxing the compiler and LSP are on my system.
It feels really wasteful to burn CPU and spin up fans every time I save a file. I find it hard to justify using 30+ GB of memory to run an LSP and compiler. I know those are tooling complaints and not really the fault of the language, but they go hand in hand. I've tried using a ctags-based workflow using vim's built in compiler/makeprg, but it's less than ideal.
I also dislike the crates.io ecosystem. I hate how crates.io requires a GitHub account to publish anything. We are already centralized around GitHub and Microsoft, why give them more power? There's an open issue on crates.io to support email based signups but it has been open for a decade.
> It feels really wasteful to burn CPU and spin up fans every time I save a file. I find it hard to justify using 30+ GB of memory to run an LSP and compiler.
Have you tried using RustRover. I've never seen it go above 2-3GiB of RAM, but I don't write the most complex of software in Rust.
> I hate how crates.io requires a GitHub account to publish anything.
You don't need Github account to publish iirc, you need it to authorize to crates.io. You can use any Git host, but your account is tied to GitHub.
Those dependencies pretty quickly reveal themselves to be complicated and heavy. I wouldn’t blame Rust for that.
I rarely need more than what workspaces and VCS based deps give me, but when I have, putting up and using a non-official registryis pretty easy.
Then you just implement Serialize and Deserialize for TypeWithDifferentSerialization.
This cover most occasional cases where you need to work around the orphan rule. And semantically, it's pretty reasonable: If a type behaves differently, then it really isn't the same type.
The alternative is to have a situation where you have library A define a data type, library B define an interface, and library C implement the interface from B for the type from A. Very few languages actually allow this, because you run into the problem where library D tries to do the same thing library C did, but does it differently. There are workarounds, but they add complexity and confusion, which may not be worth it.
The gotcha is what happens when TypeWithSomeSerialization is not something you’re using directly but is contained within SomeOtherTypeWithSomeSerialization which you are using directly. Then things get messy.
We can't say with certainty how an unspecified in-the-future library might work, so I'm going to use serde as a stand-in.
You can implement `Serialize` for a wrapper type and still serialize `SomeOtherTypeWithSomeSerialization` (which might be used by the type being wrapper directly or indirectly) differently. It might not be derivable, of course, but "I don't want the default" sort of makes that a given.
I think a lot of developers look at Typescript and come away thinking that a static type system is something you can retrofit onto any language. These devs ask why anyone would still want to use a dynamically typed language, as though static typing is something that can be had for free. But the reality is that a robust type system ends up profoundly shaping the design of a language, and introduces these sorts of thorny design questions, with each option bringing its own tradeoffs and limitations.
We want our languages to make it easy to write correct programs. And we want our languages to make it hard to write incorrect programs. And trying to have both at once is very difficult.
I’m not convinced that the problem is actually a problem. Suppose someone writes a type PairOfNumbers with a couple fields. The author did not define a serialization. You use it in another type and want it to serialize it as:
{ "a": 1, "b": 2 }
I use it and want to serialize it as:
[ 1, 2 ]
What we’re doing is fine. You should get your serialization and I should get mine. But if either of us declares, process-wide, that one of us has determined the One True Serialization of PairOfInts, I think we are wrong.
Sure, maybe current Rust and current serde make it awkward to declare non-global serializers, but that doesn’t mean that coherence is a mistake.
> What we’re doing is fine. You should get your serialization and I should get mine. But if either of us declares, process-wide, that one of us has determined the One True Serialization of PairOfInts, I think we are wrong.
Well, fine, but then you need to actually implement a module system or something. Currently trait impls are program-wide, and if you say that you're not allowed to make global impls of a trait then that's the same as saying you're not allowed to implement traits at all.
Rust’s orphan rule has the property that there is no spooky action at all distance in terms of program semantics. If I write a library, my library behaves the same way regardless of whether the main program imports a different library.
In any case, the OP’s proposed “incoherent” scheme actually is a module system of sorts for conflicting trait impls, and it seems about right for something like serialization.
Perhaps that's fine in the particular case of serialization, but that line of thinking breaks down at more fundamental operations like `PartialEq` or `Hash`. Having a different definition for equality fundamentally breaks the program if the two versions ever mix. On the flip-side, it's important that the author is allowed to declare the "correct" way to do something, e.g. in a smart pointer crate where the safety of the program relies on a correct implementation. If traits weren't program-wide, you're just kicking the bucket down the road from the library author having to do define every trait impl to every user defining every trait impl, which is even worse imo
I feel like encapsulation and composition are in strong tension, and this is one place where it boils over.
I've written a decent bit of Rust, and am currently messing around with Zig. So the comparison is pretty fresh on my mind:
In Rust, you can have private fields. In Zig all fields are public. The consequences are pretty well shown with how they print structs:
In Rust, you derive Debug, which is a macro that implements the Debug trait at the definition site. In Zig, the printing function uses reflection to enumerate the provided struct's fields, and creates a print string based on that. So Rust has the display logic at the definition site, while Zig has the logic at the call site.
It's similar with hash maps: in Rust you derive/implement the Hash and PartialEq trait, in Zig you provide the hash and eq function at the call site.
Each one has pretty stark downsides:
Zig - since everything is public, you can't guarantee that your invariants are valid. Anyone can mess around with your internals.
Rust - once a field is private (which is the convention), nobody else can mess with the internals. This means outside modules can't access internal state, so if the API is bad, you're pretty screwed.
Honestly, I'm not sure if there is a way to resolve this tension.
EDIT: one more thought: Zig vs Rust also shows up with how object destruction is handled. In Rust you implement a Drop trait, so each object can only have one way to be destroyed. In Zig you use defer/errdefer, so you can choose what type of destructor runs, but this also means you can mess up destruction in subtle ways.
Is this really that big a downside? It encourages good APIs.
The alternative of everything being public is the kind of feature that quickly becomes a big disadvantage in larger systems and teams, where saying “just don’t footgun yourself” is not a viable strategy. If there’s a workaround to achieve some goal, people will use it, and you end up with an unmaintainable mess. It’s why languages whose names start with C feature so prominently on CVE lists.
There are always corner cases where you might need to do something differently.
I had three memorable cases in my career:
1. Python 2.6x had a a stdlib bug where windows event logging did crash the process when the user had some rights set differently. Fix submitted but for the meantime we simply overwrote the private function and could ship.
2. Also python: scikit-learn had a primitive "print everything" strategy, but we need to get it into a logging framework. We overwrote their print wrapper and could ship.
3. In C#, a third party lib insisted on dumping a result to a file. We used reflection to get that as a stream.
All three are not ideal - but I think having escape hatches is important. I also think private/public is overrated. Having it as a signal is ok. Forbidding access to privates is too strong.
Three cases in your career doesn't sound like a strong counterargument to me.
I agree that escape hatches can be a good idea, though. But they should be very controlled, e.g. requiring annotations in the code, something that can be reported on by automated tooling and that can't just be done inconspicuously.
Personally, I am comfortable with Pythons "linter warning and we are all adults here" - it works well and I have never seen that somebody cried "I overwrote this private method and after an upgrade it did not work!".
.Net allows it via reflection and considering that .Net Frameworks could run untrusted code it was okay that it was forbidden out of the box (since reflection was forbidden for untrusted code). But in the current world, where untrusted code does not really exist anymore? It's just legacy cruft.
The problem is it only takes one bad or incomplete API needed for your specific use case. I ran into this a lot when I used cpal. For example, the data stream enum type (i16, u8, f32, etc) didn't have Hash or Eq derived, so I had to create a wrapper class for the data stream type. But, the type was marked non exhaustive, so I wouldn't be able to tell if my wrapper would get out of sync with theirs. It was a pain to work around.
In other cases, I couldn't work around, so I had to vendor some things. I ended up implementing my own graph library, because the existing one wouldn't let me reach into change some invariants for undo/redo. Which I mean, fair enough if that's what's needed, but it's a real pain to reimplement something because the API was incomplete. And of course, now if something from another library needs petgraph, I'd have to convert my graph to its graph.
So yes, in theory, if we had great APIs this wouldn't be a problem. Unfortunately, APIs are always a work in progress, and sometimes need escape hatches in order to send values between libraries.
> Zig vs Rust also shows up with how object destruction is handled.
I often hear critiques of Drop being less efficient for anything Arena-like, where batch destruction would be better, and holding that as the reason defer is a better approach. What is not mentioned there is that there's nothing stopping you from having both. In Rust you can perform batch destruction by having additional logic (it might require control of the container and its contents' types for easiest implementation), but the default behavior remains sane.
That's fair, since you can leak the box. I will say though it's not as ergonomic as defer, as defer handles all exits from the scope, where it's trickier to juggle destructors. Though on further thought, I suppose the arena can have Drop.
Here I'm able to swap out std.MultiArrayList's backing to be backed by virtual memory, and correctly clean it up. I'm not sure you can really do that with Rust, barring making custom data structures for everything.
there is no good way to handle colliding implementations. Both from parallel crates and due to changes over time.
Without it you can have many many additional forms of breakage. Worse you can have "new" breakage between two 3rd party crates without either of them changing due to some impl in a common ancestor changing (e.g. std) and this affecting two wild card implementations in each, now leading to an overlap.
When you have an overlap there are two options:
- fail compilation, but as mentioned this could be caused by a non breaking change in std in two in theory unrelated 3rd party dependencies
- try to choose one of the implementations. But that now gets very messy in multiple points: a) Which impl. to choose when. b) The user knowing which is chosen. c) Overlap with interactions with stuff like double dispatch, thread local variables, and in general side effects. The issues here are similar to specialization (and part why that is stuck in limbo), but a magnitude more complex as specialization is only (meant) for optimizations, while this can be deeply different behavior. Like `foo.bar()` with the same `use Bar as _;` might in one context return an `u32` and in another a `String`
In many other ecosystems it's not uncommon to run into having issues where certain libraries can't be used together at all. In rust that is close to not a thing (no_mange collisions and C dependencies are the only exception I can think of).
Similar, in my experience the likely hood of running into unintended breaking changes is lower in the rust ecosystem then e.g. python or js, that is partially due to coherence rules forcing a more clean design.
Also people are forced to have a somewhat clean dependency tree between crates in ways not all languages requires. This can help with incremental builds and compiler time, a area rust needs any help it can get. (As a side note, clean dependency structures in your modules can (sometimes) help will rust better parallelizing code gen, too.)
So overall it I think it's good.
Through it can be very annoying. And there is some potential for improvement in many ways.
---
EDIT: sorry some keyboard fat-fingering somehow submitted a half written response without me pressing enter...
In my experience using newtypes like this causes a constant shuffle between the original type and the newtype.
If a library exposes Foo and I wrap it in MyFoo implementing some trait, I need to convert to MyFoo everywhere the trait is needed and back to Foo everywhere the original type is expected.
In practice this means cluttering the code with as_foo and as_myfoo all over the place.
You could also impl From or Deref for one direction of the conversion, but it makes the code less clear in my opinion.
One strategy I like is to declare “view” types for serialization and deserialization, because you’re going to be doing that anyway if your serialized format is meant to be compatible across versions anyway.
Serde also comes with a bunch of attributes and features to make it easy to short-circuit this stuff ad hoc.
I know this only solves the serialization use case, but that seems to be where most people run into this.
honestly in my experience it rarely matters (if you care about stable APIs) as most types you want to have at an API boundary are written (or auto generated) by you
this leaves a few often small types like `DateTime<Utc>`, which you can handle with serde serialization function overwrite attributes or automatic conversions not even needing new types (through some of this attributes could be better designed)
serde is not perfect but pretty decent, but IMHO the proc macros it provides need some love/a v2 rewrite, which would only affect impl. code gen and as such is fully backward compatible, can be mixed with old code and can be from a different author (i.e. it doesn't have the problem)
Anyway that doesn't make the problem go away, just serialization/serde is both the best and worst example. (Best as it's extremely wide spread, "good enough" but not perfect, which is poison for ecosystem evolution, worst as serialization is enough of a special case to make it's best solution be potentially unusable to solve the generic problem (e.g. reflections)).
one you solve when initially writing code (so you can properly account for it and control it)
instead of a problem which can blow up when you update a package for a very pressing security fix
in the end it a question what is more important, stability or the option to monkey patch functionality into your dependencies without changing them
and given that you can always non-monkey patch crates (rust makes vendoring dep. relatively easy in case upstream doesn't fix things) I prefer the stability aspect (through if you do patch crates you re-introduce many of the issues in a different place, with the main difference of there being a chance to upstream you changes)
I've never once pulled in a new dependency and had the program fail to compile just by virtue of that dependency being present [because both my code and the new code both impl'd the same trait on the same type in some other code]. Because that can't happen because of coherence. (Right?)
It's so easy to forget about the problems we don't have because of the (good)
choices people have made in the past.
> Because that can't happen because of coherence. (Right?)
yes
Through you still can run into it when unsafe is involved, e.g. C FFI/no_mange or ASM with non-mangled labels as they are globally unique. Through IMHO, it's not a common problem and has ways to make it very unlikely for the projects where it matters.
In the end if you pull in C-FFI code (or provide it) you do ope yourself up to C ABI specific problems.
The blog post gives an example of how the current approach makes dependencies incompatible.
> if someone publishes an alternative to serde (say, nextserde) then all crates which have added support for serde also need to add support for nextserde. Adding support for every new serialization library in existence is unrealistic
If I use serde, I cannot use a crate that only implements nextserde.
If I want to use nextserde, I lose the ability to use all the crates that only implement serde.
To me, the correct solution to the problem of being tied to one ecosystem crate for utility features like serialization or logging is reflection / comptime. The problem is not the orphan rule, it’s that Rust needs reflection a lot more than a dynamically-typed language does, and it should have been added a long time ago. (It’s in development now, but it will most likely be years before it ships in a stable version.)
"Note that nonbinary crates still obey the orphan rules."
I find it slightly humorous that this sentence contains three words which would be understood completely differently by the majority of the English-speaking population.
I’m not sure I fully understand but this seems to be the kind of problem that Ocaml functors solve. You program against an interface (signature) and you supply a concrete implementation (structure) when you want to run it. You can use different implementations in different parts of your application.
So maybe do something similar in Rust by expanding how you import and export modules?
I don't think explicit naming of impls is wise. They will regularly be TraitImpl or similar and add no real value. If you want to distinguish traits, perhaps force them to be within separate modules and use mod_a::mod_b::<Trait for Type> syntax.
> An interesting outcome of removing coherence and having trait bound parameters is that there becomes a meaningful difference between having a trait bound on an impl or on a struct:
I don't think fully-qualified paths are enough on their own. You also need some way to designate that an impl is symbolically unique and has to be referred to by path. Otherwise, you still end up with a problem where the compiler doesn't know which implementation to use unless you precisely name it.
You depend on crates A and B. A impls Foo for Bar. You pass an instance of Bar to a function that accepts `impl Foo`. You are happy. Later crate B adds an impl of Foo for Bar. Clearly _at least_ one of these must be an orphan impl, but both could be. Suddenly it's ambiguous which implementation of Foo you're talking about, so you break because B added an impl.
There are many potential problems of this flavor with letting any `impl Trait for Type` be an orphan impl and then referenced by path. What happens, for example, if an impl that was an orphan impl in one version of A becomes a coherent impl in a later version of A?
I think there has to be special syntax for named/path-referenced/symbolic impls, even if the impl does not have an identifier name, so that the compiler can know "this impl only resolves if you tell me _specifically this impl_" and the impl provider has a way to create a solid consumer contract about how to use that impl in particular.
Also, not having an identifier name would mean you can't have different impls of Foo for Bar in the same module. That's probably not a limitation anyone would care about, but it's there.
Using the mod name would give it a unique name, just implicitly through the module, so I don't see the issue, unless you wanted to allow a single module to allow multiple impls of the same item.
I also don't see an issue with having multiple impls of the same trait, as long as they don't provide duplicate items inside a module. I often do multiple impl blocks to break up larger logic and organize docs, though this is generally not for trait impls, but I don't see why it couldn't be.
Let me be clear though, I'm not saying this is the best path forward on the coherence/orphan situation necessarily, just a minor critique of the blog posts position. This is a famously tricky issue, and I suspect there is no silver bullet here. Though I have always wanted some way to add flexibility to the orphan rule.
> You depend on crates A and B. A impls Foo for Bar. You pass an instance of Bar to a function that accepts `impl Foo`. You are happy. Later crate B adds an impl of Foo for Bar. Clearly _at least_ one of these must be an orphan impl, but both could be. Suddenly it's ambiguous which implementation of Foo you're talking about, so you break because B added an impl.
I think that's fine. Same as what happens if B adds a new function with the same name as a function in A that you were using unqualified.
> What happens, for example, if an impl that was an orphan impl in one version of A becomes a coherent impl in a later version of A?
I don’t think Rust needs this; Rust has done great for the last decade with the coherence rules it has. I am glad to not have to worry about this, and to not have to worry about any of the downstream problems (like linker errors) that coherence structurally eliminates.
The Rust ecosystem does a lot of what I like to call "inverse dependency injection".
If a Rust library needs support for TLS, typically that library implements a feature for each existing TLS backend, and keeps first-class integration which each one. The obvious thing would be to have a TLS Trait, and have each TLS library implement that trait (i.e.: dependency injection).
Because of to the orphan rule, such a trait would likely have to be declared in a small self-contained library, and each TLS library would implement that trait. I don't see any obvious impediment (aside from the fact that all TLS implementations would have to expose the same API and set of behaviours), but for some reason, the Rust ecosystem has taken the path of "every library has first-class integration with every possible provider".
This makes it really tricky to build libraries which rely on other libraries which rely on a TLS library, because your consumers can't easily select, for example, which TLS implementation to use. Libraries end up having lots of feature flags which just propagate feature flags to their dependencies.
The article itself covers the specific reasons that has led to that exact problem and the potential solutions available in the ecosystem with their various trade-offs.
That locks users into an ecosystem that may never evolve, which can be fine but doesn't really solve one of the core issues the author was describing. It forces the ecosystem to depend on the oldest and most incumbent crates, rather than newer ones which might be better in some ways.
Great write up of a problem that I'm glad Golang sidesteps
The problem with this is that it's systemic and central to Rusts trait-based ecosystem composition.
Go’s has a version but it's much smaller and more local. In Go, consumer-defined structural interfaces remove most of the pressure that causes the Rust problem in the first place which is producer led.
coherence rules are one of those things that seem annoying until you maintain a library others depend on. orphan rules saved me from some nasty diamond dependency situations. the frustration is real when you just want to impl Display for some foreign type though — end up with a newtype wrapper every time and hate it.
Tangent to the topic: One of the great things about Go is that the Go team goal is to have a great developer experience. As a result, they try to bundle common third party libraries (mux, zap) into the standard library. For example, they offered an http server, but due to lacking features community packages offered convenience. The Go team used those libraries as a reference to what people wanted, and addrd a performant and simple http routing in the standard library[1].
From that link:
> We made these changes as part of our continuing effort to make Go a great language for building production systems. We studied many third-party web frameworks, extracted what we felt were the most used features, and integrated them into net/http. Then we validated our choices and improved our design by collaborating with the community in a GitHub discussion and a proposal issue. Adding these features to the standard library means one fewer dependency for many projects. But third-party web frameworks remain a fine choice for current users or programs with advanced routing needs.
Note the use case - someone wants to have the ability to replace a base-level crate such as serde.
When something near the bottom needs work, should there be a process for fixing it, which is a people problem? Or should there be a mechanism for bypassing it, which is a technical solution to a people problem? This is one of the curses of open source.
The first approach means that there will be confrontations which must be resolved. The second means a proliferation of very similar packages.
This is part of the life cycle of an open source language. Early on, you don't have enough packages to get anything done, and are grateful that someone took the time to code something.
Then it becomes clear that the early packages lacked something, and additional packages appear. Over time, you're drowning in cruft. In a previous posting, I mentioned ten years of getting a single standard ISO 8601 date parser adopted, instead of six packages with different bugs.
Someone else went through the same exercise with Javascript.
Go tends to take the first approach, while Python takes the second. One of Go's strengths is that most of the core packages are maintained and used internally by Google. So you know they've been well-exercised.
Between Github and AI, it's all too easy to create minor variants of packages. Plus we now have package supply chain attacks. Curation has thus become more important. At this point in history, it's probably good to push towards the first approach.
It's a social problem that's created by a technical problem.
In many languages, if you want to integrate package A with package B, you can make and share a package AB, which people can reuse. That scales, and facilitates reuse, and avoids either package having to support everything.
In Rust, if the integration involves traits, integration between package A and package B must happen either in A or in B. That creates a scaling problem, and a social problem.
Other than duck-typed languages (and I count Go as basically that), which languages actually provide this feature?
AFAIK, it’s not really very common to be able to extend foreign types with new interfaces, especially not if you own neither.
C++ can technically do it using partial specialization, but it’s not exactly nice, and results in UB via ODR violation when it goes wrong (say you have two implementations of a `std::hash` specialization, etc.). And it only works for interfaces that are specifically designed to be specialized this way - not for vanilla dynamic dispatch, say.
> Other than duck-typed languages (and I count Go as basically that), which languages actually provide this feature?
There are only like 3 significant languages with trait-based generics, and both the other ones have some way of providing orphan instances (Haskell by requiring a flag, Scala by not having a coherence requirement at all and relying on you getting it right, which turns out to work out pretty well in practice).
More generally it's an extremely common problem to have in a mature language; if you don't have a proper fix for it then you tend to end up with awful hacks instead. Consider e.g. https://www.joda.org/joda-time-hibernate/ and https://github.com/FasterXML/jackson-datatype-joda , and note how they have to be essentially first party modules, and they have to use reflection-based runtime registries with all the associated problems. And I think that these issues significantly increased the pressure to import joda-time into the JVM system library, which ultimately came with significant downsides and costs, and in a "systems" language that aims to have a lean runtime this would be even worse.
> Scala is interesting. How do they resolve conflicts?
If there are multiple possible instances you get a compilation error and have to specify one explicitly (which is always an option). So you do have the problem of upgrading a dependency and getting a compilation error for something that was previously fine, but it's not a big deal in practice - what I generally do is go back to the previous version and explicitly pass the instance that I was using, which is just an IDE key-combo, and then the upgrade will succeed. (After all, it's always possible to get a conflict because a library you use added a new method and the name conflicted with another library you were using - the way I see it this is essentially the same thing, just with the name being anonymous and the type being the part that matters)
You also theoretically have the much bigger problem of using two different hashing/sorting/etc. implementations with the same datastructure, which would be disastrous (although not an immediate memory corruption issue the way it could be in Rust). But in practice it's just not something I see happening, it would take a very contrived set of circumstances to encounter it.
> (although not an immediate memory corruption issue the way it could be in Rust)
Just to note, all of Rust's standard container types are designed such that they guarantee that buggy implementations of traits like `Hash` and `Ord` do not result in UB - just broken collections. :-)
C# does not support adding interfaces to foreign types. It does support extension classes to add methods and properties to a type, but nothing that adds fields or changes the list of interfaces implemented by a type. Rust supports this as well, because you can use traits this way.
Dependency injection is a popular solution for this problem, and you can do that as well in Rust. It requires (again) that the API is designed for dependency injection, and instead of interfaces and is-a relationships, you now have "factories" producing the implementation.
This is interesting but I wonder if you would accept that this also has the downside of moving at the speed of humans.
In a situation where you're building, I find the orphan rule frustrating because you can be stuck in a situation where you are unable to help yourself without forking half of the crates in the ecosystem.
Looking for improvements upstream, even with the absolute best solutions for option 1, has the fundamental downside that you can't unstick yourself.
This is also where I find it surprising that this article doesn't mention Scala at all. There are MANY UX/DX challenges with the implicit and witness system in Scala, so I would never guess suggest it directly, but never have I felt more enabled to solve my own problems in a language (and yes the absolute most complex, Haskell-in-Scala libraries can absolutely an impediment to this).
With AI this pace difference is even more noticeable.
I do think that the way that Scala approaches this by using imports historically was quite interesting. Using a use statement to bring a trait definition into scope isn't discussed in any of these proposals I think?
The problem is existentials, or rather the existence of existentials without the ability to explicitly override them. Even in Haskell, overriding typeclass instances requires turning off orphan checks, which is a rather large hammer.
So once you've identified this, now you might consider the universe of possible solutions to the problem. One of those solutions might be removing existentials from your language; think about how Scala would work if implicits were removed (I haven't used Scala 3, maybe this happened?). Another solution might be to decouple the whole concept of "existential implementations of typed extension points" from libraries (or crates, or however you compile and distribute code), and require bringing instances into scope via imports or similar.
Two things are true for sure, though: libraries already depend on the current behavior, whether that makes sense or not; and forcing users to understand coherence (which instance is used by which code) is almost always a giant impediment to getting users to like your language. Hence, "orphan rules", and why everyone hates Scala 2 implicits.
That said, I would love to see a solution in my favorite class of solution: where library authors can use and benefit from this, but the average user doesn't have to notice.
I tend to think that the non-existential Scala system was _so close_, and that if you _slightly_ tweaked the scoping rules around it, you could have something great.
For example, if - as a user - I could use `.serialize(...)` from some library and it used _their_ scoped traits by default, but if I _explicitly_ (named) imported some trait(s) on my side, I could substitute my own, that'd work great.
You'd likely want to pair it with some way of e.g. allowing a per-crate prelude of explicit imports that you can ::* import within the crate to override many things at once, but... I think that with the right tweaks, you could say 'this library uses serde by default, but I can provide my own Serializer trait instead... and perhaps, if I turn off the serde Cargo feature, even their default scoped trait disappears'.
That was my first thought! I never had this problem with Scala (2.x for me, but I guess there's similar syntax/concepts in 3).
The article author does talk about naming trait impls and how to use them at call sites, but never seems to consider the idea that you could import a trait impl and use it everywhere within that scope, without extra onerous syntax.
Does this still solve the "HashMap" problem though? I guess it depends on when the named impl "binds". E.g. the named Hash impl would have to bind to the HashMap itself at creation, not at calls to `insert()` or `get()`. Which... seems like a reasonable thing?
> When something near the bottom needs work, should there be a process for fixing it, which is a people problem? Or should there be a mechanism for bypassing it, which is a technical solution to a people problem?
I don't think it's a people problem in the way we usually talk about the folly of creating technical solutions to people problems.
If something like serde is foundational, you simply can't radically change it without causing problems for lots and lots of people. That's a technical problem, not a people problem, even if serde needs radical change in order to evolve in the ways it needs to.
But sure, ok, let's imagine that wasn't the case. Let's say some new group of people decide that serde is lacking in some serious way, and they want to implement their changes. They can even do so without breaking compatibility with existing users of the crate. But the serde maintainers don't see the same problems; in fact, they believe that what this new group wants to do will actively cause more problems.
Neither group of people even needs to be right or wrong. Maybe both ways have pluses and minuses, and choosing just depends on what trade offs you value more. Neither group is wrong about wanting to either keep the status quo or make changes.
This is actually a technical problem: we need to find a way to allow both approaches coexist, without causing a ton of work for everyone else.
And even if we do run into situations where things need fixing, and things not getting fixed is a people problem, I'd argue for this particular sort of thing it's not only appropriate but essential that we have technical solutions to bypass the people problems. I mean, c'mon. People are people. People are going to be stubborn and not want change. Ossification is a real thing, and I think it's a rare project/organization that's able to avoid it. Sure, we could refuse to use technical workarounds when it's people we need to change, but in so many cases, that's just running up against a brick wall, over and over. Why do that to ourselves? Life is too short.
Having said that, I totally agree that there are situations where technical workarounds to people problems can be incredibly counter-productive, and cause more problems than they solve (like, "instead of expecting people to actually parent their kids, force everyone to give up their privacy for mandatory age verification; think of the children!"). But I don't think this is one of them.
Re: specialization and the comptime/reflection initiative
Since they allow observing whether a trait is implemented or not in the current crate they would probably become unsound if impls can be declared in downstream crates. They are a partial solution but also make other solutions harder to implement soundly (and viceversa)
Also it's sort of amazing how few people modify their tools and remove the objectionable bits.
For example, Java. Checked exceptions. Everyone hates checked exceptions. They're totally optional. Nothing in the JVM talks about a checked exception. Patch javac, comment out the checked exception checker, and compile. Nothing goes wrong. You can write Java and not deal with checked exceptions.
Likewise, you can modify rustc and make it not enforce the orphan rule.
Too many people treat their tools as black boxes and their warts as things they must tolerate and not things they can fix with their own two hands without anybody's permission.
As a non Rust man, how real are the problems in this article? Does it show up in real word or is it just a edge case? I only program in C17, C++ as C with classes and C#. Anyone can give me a good read what Traits even are?
> As a non Rust man, how real are the problems in this article?
Real, but of more concern to folks designing widely-used libraries than to folks using said libraries.
> Anyone can give me a good read what Traits even are?
You can think of traits as analogous to interfaces in OOP languages (i.e. pure virtual abstract classes in C++ terminology).
They just define a set of methods that types can implement to conform to the trait, and then consumers can treat implementing types as if they were the trait.
The major differences are: traits are implemented outside the actual type implementation, so arbitrary trait implementations can be added after the type has been written (this is why we need coherence), and rust uses traits as compile-time bounds for generics (templates).
Another imprecise analogy would be to see traits as operators.
You decide to define the operator "serde::serialize" for "MyType" but then your are stuck because you can't override or select different operators for "MyType" because only one can exists.
That's a regular yet not super common issue with traits (and this is not exclusive to Rust).
It's quite irritating because you wouldn't expect this from languages with this degree of modularity.
I've been worried about this before and the problem is real. I don 't know who maintains serde but if that gets hacked its gonna be an epic supply chain attack
Serde is maintained by dtolnay, who is a very influential figure in Rust mainly through his library development. Serde, syn, anyhow etc end up being pulled in as dependencies to nearly every Rust crate. If his account was compromised, the attack surface is essentially every single other Rust crate... not ideal
Similar but not exactly the same as named impls, I'd really like to see a language handle this by separating implementing a trait from making a particular existing implementation the implicit default. Orphan rules can apply to the latter, but can be overriden in a local scope by any choice of implementation.
This is largely based on a paper I read a long time ago on how one might build a typeclass/trait system on top of an ML-style module system. But, I suspect such a setup can be beneficial even without the full module system.
Language changes could help for sure. There’s a library implementation we can use right now though: https://facet.rs/ Basically a derive macro for reflection. Yeah it’s one (more) trait to derive on all your types but then users can use that to do reflection or pretty printing or diffing or whatever they want.
It is fundamentally difficult to have an “ecosystem”.
Would much rather see a bunch of libraries that implement everything for a given use case like web-dev, embedded etc.
Unfortunately this is hard to do in rust because it is hard to implement the low level primitives.
Language’s goal should be to make building things easier imo. It should be simple to build a serde or a tokio.
From what I have seen in rust, people tend to over-engineer a single library to the absolute limit instead just building a bunch of libraries and moving on.
As an example, if it is easy to build a btreemap then you don’t have to have a bunch of traits from a bunch of different libraries pre-implemented on it. You can just copy it, adapt it a bit and move on.
Then you can have a complete thing that gives you everything you need to write a web server and it just works
So what I mean is, having a big library that implements the whole problem is better. Because then each part of that library is simple. Then I can copy paste some parts and change some others to create an alternative library. And it is better for the user of the thing because it is simple.
Having everything compatible with everything else and having everything implement every case means every individual part is over-complicated. So it is bad no matter how you combine it together.
I never understood why Rust couldn't figure this shit out. Scala did.
> If a crate doesn’t implement serde’s traits for its types then those types can’t be used with serde as downstream crates cannot implement serde’s traits for another crate’s types.
You are allowed to do this in Scala.
> Worse yet, if someone publishes an alternative to serde (say, nextserde) then all crates which have added support for serde also need to add support for nextserde. Adding support for every new serialization library in existence is unrealistic and a lot of work for crate authors.
You can easily autoderive a new typeclass instance. With Scala 3, that would be:
trait Hash[A]:
extension (a: A) def hash: Int
trait PrettyPrint[A]:
extension (a: A) def pretty: String
// If you have Hash for A, you automatically get PrettyPrint for A
given autoDerive[A](using h: Hash[A]): PrettyPrint[A] with
extension (a: A) def pretty: String = s"<#${a.hash.toHexString}>"
> Here we have two overlapping trait impls which specify different values for the associated type Assoc.
trait Trait[A]:
type Assoc
object A:
given instance: Trait[Unit] with
type Assoc = Long
def makeAssoc: instance.Assoc = 0L
object B:
given instance: Trait[Unit] with
type Assoc = String
def dropAssoc(a: instance.Assoc): Unit =
val s: String = a
println(s.length)
@main def entry(): Unit =
B.dropAssoc(A.makeAssoc) // Found: Playground.A.instance.Assoc Required: Playground.B.instance².Assoc²
Perhaps I'm insufficiently caffeinated, but isn't the author describing the expression problem? That basically nails what type classes are for (in Scala and elsewhere), no?
Does the author expect incumbents to relax language rules that grant exorbitant privilege to incumbents? The orphan rule is up there with error handling in ways Rust is a screwed up language that appeals to people who don't know what they're missing. Other systems languages aren't weak in these ways.
Could you elaborate on that error handling part? To me, Rust is the only sane language I've worked with that has error-like propagation, in that functions must explicitly state what they can return, so that you don't get some bizarre runtime error thrown because the data was invalid 15 layers deeper
I don't know what quotemstr was specifically talking about, but here's my own take.
The ideal error handling is inferred algebraic effects like in Koka[1]. This allows you to add a call to an error-throwing function 15 layers down the stack and it's automatically propagated into the type signatures of all functions up the stack (and you can see the inferred effects with a language server or other tooling, similar to Rust's inferred types).
Now, how do you define E4, E5 and E6? The "correct" way is to use sum types, i.e., `enum E4 {E1(E1), E2(E2)}`, `enum E5 {E1(E1), E3(E3)}` and `enum E6 {E1(E1), E2(E2), E3(E3)}` with the appropriate From traits. The problem is that this involves a ton of boilerplate even with thiserror handling some stuff like the From traits.
Since this is such a massive pain, Rust programs tend to instead either define a single error enum type that has all possible errors in the crate, or just use opaque errors like the anyhow crate. The downside is that these approaches lose type information: you no longer know that a function can't return some specific error (unless it returns no errors at all, which is rare), which is ultimately not so different from those languages where you have to guard against bizarre runtime errors.
Worse yet, if f1 has to be changed such that it returns 2 new errors, then you need to go through all error types in the call stack and flatten the new errors manually into E4, E5 and E6. If you don't flatten errors, then you end up rebuilding the call stack in error types, which is a whole different can of worms.
Algebraic effects just handle all of this more conveniently. That said, an effect system like Koka's isn't viable in a systems programming language like Rust, because optimizing user-defined effects is difficult. But you could have a special compiler-blessed effect for exceptions; algebraic checked exceptions, so to speak. Rust already does this with async.
I will never stop hating on the orphan rule, a perfect summary of what’s behind a lot of rust decisions. Purism and perfectionism at the cost of making a useful language, no better way to torpedo your ecosystem and make adding dependencies really annoying for no reason. Like not even a —dangerously-disable-the-orphan-rule, just no concessions here.
I think there are legitimate criticisms of Rust that fall in this category, but the orphan rule ain’t it.
In most other languages, it is simply not possible to “add” an interface to a class you don’t own. Rust let’s you do that if you own either the type or or the interface. That’s strictly more permissive than the competition.
The reasons those other languages have for not letting you add your interface to foreign types, or extend them with new members, are exactly the same reasons that Rust has the orphan rule.
The orphan rule is annoying. It's also one of the few things stopping trait impls from turning crate boundaries into a knife fight, where two deps both decide they own the same foreign type and the compiler has no sane answer once both impls are in the graph.
Rust pays for coherence up front with wrapper types and boilerplate, which is ugly, but the alternative is the kind of ambient monkeypatching that makes APIs hard to reason about once a codebase gets large. A narrow escape hatch might be worth trying, but a global 'disable it' switch sounds like repo poison.
> the compiler has no sane answer once both impls are in the graph
but the user could provide an answer.
Julia allows something called "piracy" which is spiritually similar to the orphan rule. and while it is strongly discouraged, the compiler allows it. and when "ambiguities" arise (here called "coherence") it becomes a social problem, not technical, to resolve. and usually package authors are pretty willing to cooperate to resolve it!
This is one of the (several?) things that make me very worried about Rust long-term. I love the language, and reach for it even when it sometimes isn't the most appropriate thing. But reading some of the made-up syntax in the "Removing Coherence" section makes my head hurt.
When I used to write Scala, I accepted the fact that I don't have a background in type/set/etc. theory, and that there were some facets of the language that I'd probably never understand, and some code that others had written that I'd probably never understand.
With a language like Rust, I feel like we're getting there. Certain GAT syntxes sometimes take some time for me to wrap my head around when I encounter them. Rust feels like it shouldn't be a language where you need to have some serious credentials to be able to understand all its features and syntax.
On the other end we have Go, which was explicitly designed to be easy to learn (and, unrelatedly, I don't like for quite a few reasons). But I was hoping that we could have a middle ground here, and that Rust could be a fully-graspable systems-level language.
Then again, for more comparison, I haven't used C++ since before they added lambdas. I wonder if C++ has some hairy concepts and syntax today on par with Rust's more difficult parts.
> I wonder if C++ has some hairy concepts and syntax today
Both better and worse.
The current version of idiomatic C++ is much cleaner, more concise, and more powerful than the version of C++ you are familiar with. You don't need C-style macros anymore. The insane template metaprogramming hacks are gone. Some important things that were problematic to express in C++ (and other systems languages to be fair) are now fully defined e.g. std::launder. C++ now has expansive compile-time programming features, which is killer for high-performance systems code, and is more expressive than Rust in important ways.
The bad news is that this was all piled on top of and in addition to the famous legacy C++ mess for backward compatibility. If you are mixing and matching ancient C++ with modern C++, you are going to have a bad time. That's the worst of all worlds.
But if you are lucky enough to work with e.g. an idiomatic C++20 code base, it is a much simpler and better language than legacy C++. I've done a few major C++ version upgrades to code bases over the years; the refactored code base was always smaller, cleaner, safer, and easier to maintain than the old version.
These problems will happen in some form for every long lived codebase. In 15 years you will have some regrets, and some of those regrets will be things that are all core APIs used everywhere in the code base and so impossible to quickly change.
Isn't it 100x worse in C++ though? Because the surface area is huge? I work on legacy apps but PHP 5.6 is not that fundamentally different from PHP 8.3.
No, it is nearly the same - most problems are your own making because of your own lack of foresight. (perfect foresight is impossible)
It's much simpler and better than it used to be but it's still pretty bad. As just one example off the top of my head consider the meaning of curly braces for initialization. There's several different things they can mean depending on the context. Good luck figuring out which set is currently in effect.
The initialization situation in C++ is indefensibly broken. It is near the top of my list of things I hate about C++.
You can mitigate it with some practices but that this is even necessary is a crime. Initialization is one of the most basic things in software development. How do you fuck it up so badly?
On a day to day basis it doesn’t cause me issues but it offends me just on principle.
Use static analyzers and move on. Almost all the complaints I see about C++ nowadays are removed by max warning levels. Set them as error.
Certainly initialization is the single most confusing feature in C++, I can give you that.
But still doable with s few patterns to remember. And warnings always max level.
I still use Eclipse CDT and its static analysis is running in real time, as you type code, which is killer. Combined with Valgrind integration, I don't see myself moving on anytime soon.
Is Eclipse CDT still good these days? Wow did not hear of it for a while. I thought C++ support was not maintained anymore.
I use CLion mostly but I never stop coming back to Emacs+LSP.
And yes, the analysis is quite competitive tbh. People often talk about this weird thing or the other in C++ but the experience is quite better than what the ISO standard strictly has to offer.
Eclipse is getting stable releases four times a year (i.e. every three months). C/C++ support is also being actively maintained and is pretty fast these days.
Eclipse is one of the rare software suites which didn't get slower as the tech evolves. Yes, it's probably heavier when compared to 20 years ago, but it starts pretty quickly and works snappily. I'm a happy camper.
If only the Go tools didn't get discontinued, but alas. KATE/BBEdit + Gopls is a pretty nifty combo on Linux/macOS.
> I wonder if C++ has some hairy concepts and syntax today on par with Rust's more difficult parts.
… … … … Unqualified name lookup has been challenging in C++ since even before C++11. Overload resolution rules are so painful that it took me weeks to review a patch simply because I had to back out of trying to make sense of the rules in the standard. There's several slightly different definitions of initialization. If you really want to get in the weeds, starting playing around with std::launder and std::byte and strict aliasing rules and lifetime rules, and you'll yearn for the simplicity of Rust.
C++ is the absolute most complex of any of the languages whose specifications I have read, and that's before we get into the categories of things that the standard just gives up on.
> starting playing around with std::launder and std::byte and strict aliasing rules and lifetime rules, and you'll yearn for the simplicity of Rust
Annotations like std::launder, lifetime manipulation, etc solve a class of problems that exist in every systems language. They inform the compiler of properties that cannot be known by analyzing the code. Rust isn't special in this regard, it has the same issues.
Without these features, we either relied on unofficial compiler-specific behavior or used unnecessarily conservative code that was safe but slower.
> Rust isn't special in this regard, it has the same issues.
This is both fundamentally true and misleading. Rust has to solve the same issues but isn't obliged to make all the same bad choices to do that and so the results are much better.
For example C++ dare not perform compile time transmutations so, it just forbids them and a whole bunch of extra stuff landed to work around that, but in Rust they're actually fine and so you can just:
That blows up at compile time because we claimed the bit pattern for the integer 2 is a valid boolean and it isn't. If we choose instead 0 (or 1) this works and we get the expected false (or true) boolean instead of a compiler diagnostic.C++ could allow this but it doesn't, rather than figure out all the tricky edge cases they just said no, use this other new thing we made.
> For example C++ dare not perform compile time transmutations
I am confused by this assertion. You can abuse the hell out of transformations in a constexpr context. The gap between what is possible at compile-time and run-time became vanishingly small a while ago.
I think your example is not illustrative in any case. Many C++ code bases work exactly like your example, enforced at compile-time. That this can be an issue is a hangover from retaining compatibility with C-style code which conflates comparison operators and cast operators. It is a choice.
C++ can enforce many type constraints beyond this at compile-time that Rust cannot, with zero effort or explicit type creation. No one should be passing ints around.
First of all mem::transmute is like bit_cast (which works perfectly fine in constexpr context), not reinterpret cast.
Second, this compiles just fine:
This fails at compile time (invalid narrowing): Note we don't need bit_cast for this example as int to bool conversions are allowed in C++.Surely "We have many different ways to do this, each with different rules" is exactly the point? C++ 20's std::bit_cast isn't necessarily constexpr by the way although it is for the trivial byte <-> boolean transmutation I mentioned here.
I see that C++ people were more comfortable with the "We have far too many ways to initialize things" examples of this problem but I think transmutation hits harder precisely because it sneaks up on you.
bit_cast and reinterpret_cast do different things: one works at the value level, the second preserves address identity (and it is problematic from an aliasing point of view).
Not sure what any of this has to do with initialization though.
FWIW, the direct translation of your rust code is:
It fails on clang for y=2 and works for y=1, exactly like rust;GCC produces UB for y=2, I don't know if it is a GCC bug or the standard actually allows this form of UB to be ignored at contexpr time.
What is the rust equivalent of reinterpret_cast and does it work at constexpr time?
edit: I guess it would be an unsafe dereference of a casted pointer. Does it propagate constants?
Firstly, that's not a direct translation because you're making two variables and I made none at all. Rust's const is an actual constant, it's not an immutable variable. We have both, but they're different. The analogous Rust for your bit cast example would make two immutable variables that we promise have constant values, maybe:
Of course this also won't compile, because the representation for 2 still isn't a boolean. If it did compile you'd also (by default) get angry warnings because it's bad style to give these lowercase names.I also don't know if you found a GCC bug but it seems likely from your description. I can't see a way to have UB, a runtime phenomenon, at compile time in C++ as the committee imagines their language. Of course "UB? In my lexer?" is an example of how the ISO document doesn't understand intention, but I'd be surprised if the committee would resolve a DR with "That's fine, UB at compile time is intentional".
I understand that "these are different things" followed by bafflegab is how C++ gets here but the whole point of this sub-thread is that Rust didn't do that, so in Rust these aren't "different things". They're both transmutation, they don't emit CPU instructions because they happen in the type system and the type system evaporates at runtime.
So this is an impedance mismatch, you've got Roman numerals and you can't see why metric units are a good idea, and I've got the positional notation and so it's obvious to me. I am not going to be able to explain why this is a good idea in your notation, the brilliance vanishes during translation.
I'm using two variables because numeric literals have the wrong type and bit_cast rejects transmutations between differently sized types.
I could have written it as x = bit_cast<bool>(char{2}), but does it really make a difference?
I don't know enough rust to know what's the difference between its const and c++ constexpr. It might not be a meaningful difference in C++.
> So this is an impedance mismatch, you've got Roman numerals and you can't see why metric units are a good idea, and I've got the positional notation and so it's obvious to me. I am not going to be able to explain why this is a good idea in your notation, the brilliance vanishes during translation.
There are plenty of rust users on HN that are capable of kind, constructive, and technically interesting conversations. Unfortunately there are a small few that will destroy any goodwill the rest of the community works hard to generate.
> I could have written it as x = bit_cast<bool>(char{2}), but does it really make a difference?
Not really, that's also a variable. We're running into concrete differences here, which is what I was gesturing at. In C++ you've got two different things, one old and one new, and the new one does some transmutations (and is usually constexpr) while the old one does others but isn't constexpr. It's not correct to say that reinterpret_cast isn't a transmutation, for example it's the recognised way to do the "I want either a pointer or an integer of the same size" trick in C++ which is exactly that. Let me briefly explain, as much to ensure it's clear in my head as yours:
In C++ we have an integer but sometimes we're hiding a pointer in there using reinterpret_cast, in Rust we have a pointer but sometimes we're hiding an integer in there using transmute [actually core::ptr::without_provenance but that's just a transmute with a safe API]. Of course the machine code emitted is identical, because types evaporate at compile time the CPU doesn't care whether this value in a register "is" a pointer or not.
Anyway, yes the issues are the same because ultimately the machines are the same, but it's not true that C++ solved these issues the only way they could be addressed, better is possible. And in fact it would surely be a disappointment if we couldn't do any better decades later. I hope that in twenty years the Rust successor is as much better.
I don't know a way to express actual constants in C++ either. If there isn't one yet maybe C++ 29 can introduce a stuttering type qualifier co_co_const to signify that they really mean constant this time. Because constexpr is a way to get an immutable variable (with guaranteed compile time initialization and some other constraints) and in C++ we're allowed to "cast away" the immutability, we can actually just modify that variable, something like this: https://cpp.godbolt.org/z/EYnWET8sT
In contrast it doesn't mean anything to modify a constant in either language, it's not a surprise that 5 += 2 doesn't compile and so likewise Rust's core::f32::consts::PI *= 2; won't compile, and if we made our own constants we can't change those either. We can write expressions where we call into existence a temporary with our constant value, and then we mutate the temporary, but the constant itself is of course unaffected if we do this.
This can be a perf footgun, you will see newcomers write Rust where they've got a huge constant (e.g a table of 1000 32-bit floating point numbers) and they write code which just indexes into the constant in various parts of their program, if the index values are known at compile time this just optimises to the relevant 32-bit floating point number, because duh, but if they aren't it's going to shove that entire table on your stack everywhere you do this, and that's almost certainly not what you intended. It's similar to how newcomers might accidentally incur copies they didn't mean in C++ because they forgot a reference.
> https://cpp.godbolt.org/z/EYnWET8sT
I'm afraid that just C++ being C++ and you are deep into UB; you can't really modify a constexpr value at runtime, and if you cast away its constness with what is effectively a const cast you are on your own. This will print "0 3" which is obviously nonsense:
The output might change according to the compiler and optimization level.You can also move the problematic sequence into a constexpr function and invoke it in a constexpr context: the compiler will reject the cast now.
Enum constants can't become lvalues, so the following also won't compile:
So I guess that's closer to the rust meaning of constant.FWIW, notoriously you could modify numeric literals in early Fortrans and get into similar nonsense UB.
edit: in the end[1] it seems that you take exception with constexpr not being prvalues in C++. I guess it was found more convenient for them to have an address [1]. That doesn't make them less constant.
[1] or at least I think you do, it is not clear to me what you have been trying to claim in this discussion.
[2] C++ will materialize prvalues into temporaries when their address is needed (and give them an unique address), I guess it was thought to be wasteful for large constexpr objects, and avoids the rust pitfall you mentioned.
> I'm afraid that just C++ being C++ and you are deep into UB
Of course, but the reason to even do this is only to illustrate that it's just another variable, nothing more. As much for myself as for you.
I've heard the stories about older languages but I assume that's not a thing on a modern Fortran and I'm sure we agree it's a bad idea.
The main thrust of this sub-thread was that languages can, and I believe Rust did, choose to solve the same issues but in a better way and so "This is too complicated in C++" doesn't translate to "It will be too complicated in every language". I think some C++ people have a version of the "End of History" nonsense, which is a nineteenth century idea. If you think noteworthy change happened after that and so history didn't end then hopefully you agree that makes no sense for general world history, and perhaps you can agree likewise C++ isn't the final apex of programming languages.
Is there a reason Rust would not (as it was done in the ‘good ole days’) index the table via pointer arithmetic from .data? Also, I’m assuming that because you are discussing new devs, that they are not making the implementation decision to place the table on the heap and using Rist’s subscript operator, which I would understand Rust not doing as default. I can not think of a reason that the table should ever be put on the stack for reading a single value, so that being the default seems an oddly pessimistic default. I could be missing something regarding how Rust handles literal data ‘written out’ into source though.
The table is on the stack because we conjured into existence a temporary, so it's exactly as if the programmer had conjured the variable as a local by hand. Suppose our table is named SOUP
However Now blah is a variable, we might need any value from SOUP, the optimiser doesn't know what values it might have - so a temporary is conjured into existence, equivalent to: You do presumably recognise that this now puts SOUP on the stack right? The temporary is equivalent, but without the explicitness.Now if you know what you're doing you would of course have one single place in your program where you do this:
And now there's an immutable global named SOUP and like "the good ole days" we don't keep writing this huge data blob to the stack only to subsequently look at a single element. But that's not the thing the noob wrote so that's not what they get."Sufficiently smart compilers" are a very gradual miracle. In theory a compiler could realise this is a good idea, in practice today I doubt you will find such a compiler, so just write explicitly that you want a single variable if that's what you want.
In my experience conversions is one of the things that maximum warning levels do excellent static analysis for nowadays. In the last 15 years I hardly had a couole problems (init vs paren initialization). All narrowing etc. is caught out of the box with warnings.
I'm not sure what you're getting at but
const bool z = (const bool)((int8_t)2);
Is perfectly valid C++.
That's a conversion, not the same. The naive equivalent to transmute would be
But reinterpret_cast isn't valid in a constexpr scope.My point is, in your exact example both reinterpret_cast and C-style casts have the exact same behavior, making the example bad. If you want to showcase a deficiency of C++, it would make sense to pick something where the difference between cast types actually matters.
> But reinterpret_cast isn't valid in a constexpr scope.
std::bit_cast is
Oh cool, and it behaves like memcpy, not like pointer aliasing! I'm stuck with C++14 at work so I missed that one.
The right strategy to use C++ efficiently is to set warnings to the maximum as errors and take the core guidelines or similar and avoid past cruft.
More often than not (except if you inherit codebases but clang has a modernize tool) most of the cruft is either avoidable or caught by analyzers. Not all.
But overall, I feel that C++ is still one of the most competitive languages if you use it as I said and with a sane build system and package manager.
Unless you're writing a compiler, you should require the author of the patch to explain why it works.
This was a patch for the compiler implementation of the changes to the standard.
> But reading some of the made-up syntax in the "Removing Coherence" section makes my head hurt.
Articles discussions new features always have difficult syntax. There have been proposals like this going on from the start.
Fortunately the language team is cognizant of the syntax and usability issues with proposals. There have been a lot of proposals that started off as very unwieldy syntax but were iterated for years until becoming more ergonomic.
I think it's more than the syntax, it's just the number of concepts you need to keep in your head to read a type signature, or a trait declaration, or whatever. There's only so much you can pack into a limited language before it becomes too much to keep in your head at once.
Most of the time over the past couple years when somebody complained about Rust getting a complex new feature, it was complex backend work that exhibited to users as the removal of restriction, making the language simpler. Example: async closures. A language that supports both async and closures but doesn't support them together is more complicated than one that supports async closures.
This feels like it should end up similar. The driving desire here is the removal of a restriction, so hopefully it ends up as an end user simplification rather than complication.
And then someone declares "this language is a mess! I'll take the good parts and create a new one without all this cruft!" and the cycle continues
I really wouldn't worry much. Over the last decade of rust, very few of the articles exploring new syntax have turned into anything controversial by the time they were merged (I can't even think of big rust syntax changes other than `impl T` in arguments etc). The singular example really is async/await and, having been quite worried about it / against it at the time, it was really nothing to be concerned with at all.
let-else was a more recently added new syntax
Indeed, but it hardly seems controversial or major. It's sort of just an extension of existing syntax, similar to `impl Trait` in argument position.
Having used Rust professionally for six years now, I share your fear. Like many of the commenters below, coherence just hasn't been a big problem for me. Maybe there are problem spaces where it's particularly painful?
How does the Rust language team weigh the benefits of solving user problems with new language features against the resulting increased complexity? When I learned Rust, I found it to be quite complex, but I also got real value from most of the complexity. But it keeps growing and I'm not always sure people working on the language consider the real cost to new and existing users when the set of "things you have to know to be competent in the language" grows.
You can look at the discussions in any of the language RFCs to see that increased complexity is one of the recurring themes that get brought up. RFCs themselves have a "how do we teach this?" section, that IMO makes or break a proposal.
Keep in mind that as time goes on, features being introduced will be more and more niche. If you could do things in a reasonable way without the new feature, the feature wouldn't be needed. That doesn't mean that everyone needs to learn about the feature, only the people that need that niche have to even know about it (as long as it is 1) it interacts reasonably with the rest of the language, 2) its syntax is reasonable in that it is either obvious what's going on or easy to google and memorable so that you don't have to look it up again and 3) it is uncommon enough that looking at a random library you won't be seeing it pop up).
Thanks for the context. That makes a lot of sense! Those three constraints seem pretty important and a useful way to think about the problem.
> I wonder if C++ has some hairy concepts and syntax today
https://tartanllama.xyz/posts/cpp-initialization-is-bonkers/
Rust opened the door to innovation in the low-level languages space, but as long as it is already the most theoretically advanced practical language there, it will always attract the audience that actually wants to push it further. I don't know if there is a way to satisfy both audiences.
A similar thing happened with C++: The fact that it had a relatively high complexity of interacting features (already 30+ years ago) that you could use to do smart things, did attract smart people with a high tolerance for complexity. And with such an audience, complexity tends to expand up to the limits of tolerance (or even a little beyond).
Rust had a better start, not the least because it wasn’t designed on top of an existing language like C++ was, but who knows what it will look like in 30 years.
I think there is: a schism. Another language, inspired, intelligible and interoperable with Rust, but with other goals, likely ease of use or surface simplicity. In my mind it would be pretty much the same as Rust, but whenever a compile error gives you a suggestion in rustc would instead compile (and at most be a warning in this hypothetical language). Migrating from Rust to this language would be changing a single setting in Cargo.toml. The other way around would be fixing a bunch of compile errors. You could use the entire crate ecosystem in a native way. This language could also serve as a test bed for features that might or might not be suitable for Rust. It can also have a more aggressive evolution schedule, meaning that it wouldn't be perma-1.x, so it can be bolder on what is attempted.
There is precedent: with type checkers like pyright you can opt into specific checks, or have a basic, standard, strict setting, each expanding the set of checks done.
How would dependencies work in this schism? E.g. if serde starts using named impls, do all dependencies have to use named impls?
I'd take `Rust with a GC and specialization` over current Rust any day.
I mean… Sure, if we’re just making stuff up, a compiler that can magically understand whatever you were trying to do and then do that instead of what you wrote, I guess that’s a nice fantasy?
But out here on this miserable old Earth I happen to think that Rust’s errors are pretty great. They’re usually catching things I didn’t actually intend to do, rather than preventing me from doing those things.
> But out here on this miserable old Earth I happen to think that Rust’s errors are pretty great. They’re usually catching things I didn’t actually intend to do, rather than preventing me from doing those things.
As it happens, you are replying to the person who made Rust's errors great! (it wasn't just them of course, but they did a lot of it)
I bow to them and thank them for their service!
Reflection syntax (C++26 I think) has made my 30+ years-of-C++ brain melt.
It's not insane, it's just ... melt-inducing.
Yeah, for me reflection and coroutines were the first changes to C++ where the implementation and use mechanics weren't immediately obvious by reading a few references. It requires a bit of proper study to wrap your head around it.
I agree. It was the first time I actually thought 'who the hell writes code like this?'
> I wonder if C++ has some hairy concepts and syntax today on par with Rust's more difficult parts
Yes, it's called "C++".
I wonder how Zig compares here
I wonder if Google's Carbon could fill that gap?
https://en.wikipedia.org/wiki/Carbon_(programming_language)
> But reading some of the made-up syntax in the "Removing Coherence" section makes my head hurt.
It's interesting to note the discrepancy between replies to this blog here and, say, lobste.rs (which is neutral to it).
Here it's very concerned about complexity, while on lobsters it's mostly about needing this feature - yesterday.
I used Rust for ~14 months and released one profitable SaaS product built entirely in Rust (actix-web, sqlx, askama).
I won't be using Rust moving forward. I do like the language but it's complicated (hard to hold in your head). I feel useless without the LSP and I don't like how taxing the compiler and LSP are on my system.
It feels really wasteful to burn CPU and spin up fans every time I save a file. I find it hard to justify using 30+ GB of memory to run an LSP and compiler. I know those are tooling complaints and not really the fault of the language, but they go hand in hand. I've tried using a ctags-based workflow using vim's built in compiler/makeprg, but it's less than ideal.
I also dislike the crates.io ecosystem. I hate how crates.io requires a GitHub account to publish anything. We are already centralized around GitHub and Microsoft, why give them more power? There's an open issue on crates.io to support email based signups but it has been open for a decade.
> It feels really wasteful to burn CPU and spin up fans every time I save a file. I find it hard to justify using 30+ GB of memory to run an LSP and compiler.
Have you tried using RustRover. I've never seen it go above 2-3GiB of RAM, but I don't write the most complex of software in Rust.
> I hate how crates.io requires a GitHub account to publish anything.
You don't need Github account to publish iirc, you need it to authorize to crates.io. You can use any Git host, but your account is tied to GitHub.
Those dependencies pretty quickly reveal themselves to be complicated and heavy. I wouldn’t blame Rust for that. I rarely need more than what workspaces and VCS based deps give me, but when I have, putting up and using a non-official registryis pretty easy.
I have used C++ for 20 years without LSP, but now I wouldn't want to go back to a plain editor.
what will you be using for your next project?
There's a well-known (and frequently encouraged) workaround for the orphan rule: Create a wrapper type.
Let's say you have one library with:
And you want to define a custom serialization. In this case, you can write: Then you just implement Serialize and Deserialize for TypeWithDifferentSerialization.This cover most occasional cases where you need to work around the orphan rule. And semantically, it's pretty reasonable: If a type behaves differently, then it really isn't the same type.
The alternative is to have a situation where you have library A define a data type, library B define an interface, and library C implement the interface from B for the type from A. Very few languages actually allow this, because you run into the problem where library D tries to do the same thing library C did, but does it differently. There are workarounds, but they add complexity and confusion, which may not be worth it.
The gotcha is what happens when TypeWithSomeSerialization is not something you’re using directly but is contained within SomeOtherTypeWithSomeSerialization which you are using directly. Then things get messy.
We can't say with certainty how an unspecified in-the-future library might work, so I'm going to use serde as a stand-in.
You can implement `Serialize` for a wrapper type and still serialize `SomeOtherTypeWithSomeSerialization` (which might be used by the type being wrapper directly or indirectly) differently. It might not be derivable, of course, but "I don't want the default" sort of makes that a given.
In that case, wrap a reference maybe?
I think a lot of developers look at Typescript and come away thinking that a static type system is something you can retrofit onto any language. These devs ask why anyone would still want to use a dynamically typed language, as though static typing is something that can be had for free. But the reality is that a robust type system ends up profoundly shaping the design of a language, and introduces these sorts of thorny design questions, with each option bringing its own tradeoffs and limitations.
We want our languages to make it easy to write correct programs. And we want our languages to make it hard to write incorrect programs. And trying to have both at once is very difficult.
I’m not convinced that the problem is actually a problem. Suppose someone writes a type PairOfNumbers with a couple fields. The author did not define a serialization. You use it in another type and want it to serialize it as:
I use it and want to serialize it as: What we’re doing is fine. You should get your serialization and I should get mine. But if either of us declares, process-wide, that one of us has determined the One True Serialization of PairOfInts, I think we are wrong.Sure, maybe current Rust and current serde make it awkward to declare non-global serializers, but that doesn’t mean that coherence is a mistake.
> What we’re doing is fine. You should get your serialization and I should get mine. But if either of us declares, process-wide, that one of us has determined the One True Serialization of PairOfInts, I think we are wrong.
Well, fine, but then you need to actually implement a module system or something. Currently trait impls are program-wide, and if you say that you're not allowed to make global impls of a trait then that's the same as saying you're not allowed to implement traits at all.
Rust’s orphan rule has the property that there is no spooky action at all distance in terms of program semantics. If I write a library, my library behaves the same way regardless of whether the main program imports a different library.
In any case, the OP’s proposed “incoherent” scheme actually is a module system of sorts for conflicting trait impls, and it seems about right for something like serialization.
Perhaps that's fine in the particular case of serialization, but that line of thinking breaks down at more fundamental operations like `PartialEq` or `Hash`. Having a different definition for equality fundamentally breaks the program if the two versions ever mix. On the flip-side, it's important that the author is allowed to declare the "correct" way to do something, e.g. in a smart pointer crate where the safety of the program relies on a correct implementation. If traits weren't program-wide, you're just kicking the bucket down the road from the library author having to do define every trait impl to every user defining every trait impl, which is even worse imo
I feel like encapsulation and composition are in strong tension, and this is one place where it boils over.
I've written a decent bit of Rust, and am currently messing around with Zig. So the comparison is pretty fresh on my mind:
In Rust, you can have private fields. In Zig all fields are public. The consequences are pretty well shown with how they print structs: In Rust, you derive Debug, which is a macro that implements the Debug trait at the definition site. In Zig, the printing function uses reflection to enumerate the provided struct's fields, and creates a print string based on that. So Rust has the display logic at the definition site, while Zig has the logic at the call site.
It's similar with hash maps: in Rust you derive/implement the Hash and PartialEq trait, in Zig you provide the hash and eq function at the call site.
Each one has pretty stark downsides: Zig - since everything is public, you can't guarantee that your invariants are valid. Anyone can mess around with your internals. Rust - once a field is private (which is the convention), nobody else can mess with the internals. This means outside modules can't access internal state, so if the API is bad, you're pretty screwed.
Honestly, I'm not sure if there is a way to resolve this tension.
EDIT: one more thought: Zig vs Rust also shows up with how object destruction is handled. In Rust you implement a Drop trait, so each object can only have one way to be destroyed. In Zig you use defer/errdefer, so you can choose what type of destructor runs, but this also means you can mess up destruction in subtle ways.
> so if the API is bad, you're pretty screwed.
Is this really that big a downside? It encourages good APIs.
The alternative of everything being public is the kind of feature that quickly becomes a big disadvantage in larger systems and teams, where saying “just don’t footgun yourself” is not a viable strategy. If there’s a workaround to achieve some goal, people will use it, and you end up with an unmaintainable mess. It’s why languages whose names start with C feature so prominently on CVE lists.
There are always corner cases where you might need to do something differently. I had three memorable cases in my career: 1. Python 2.6x had a a stdlib bug where windows event logging did crash the process when the user had some rights set differently. Fix submitted but for the meantime we simply overwrote the private function and could ship. 2. Also python: scikit-learn had a primitive "print everything" strategy, but we need to get it into a logging framework. We overwrote their print wrapper and could ship. 3. In C#, a third party lib insisted on dumping a result to a file. We used reflection to get that as a stream.
All three are not ideal - but I think having escape hatches is important. I also think private/public is overrated. Having it as a signal is ok. Forbidding access to privates is too strong.
Three cases in your career doesn't sound like a strong counterargument to me.
I agree that escape hatches can be a good idea, though. But they should be very controlled, e.g. requiring annotations in the code, something that can be reported on by automated tooling and that can't just be done inconspicuously.
Personally, I am comfortable with Pythons "linter warning and we are all adults here" - it works well and I have never seen that somebody cried "I overwrote this private method and after an upgrade it did not work!". .Net allows it via reflection and considering that .Net Frameworks could run untrusted code it was okay that it was forbidden out of the box (since reflection was forbidden for untrusted code). But in the current world, where untrusted code does not really exist anymore? It's just legacy cruft.
The problem is it only takes one bad or incomplete API needed for your specific use case. I ran into this a lot when I used cpal. For example, the data stream enum type (i16, u8, f32, etc) didn't have Hash or Eq derived, so I had to create a wrapper class for the data stream type. But, the type was marked non exhaustive, so I wouldn't be able to tell if my wrapper would get out of sync with theirs. It was a pain to work around.
In other cases, I couldn't work around, so I had to vendor some things. I ended up implementing my own graph library, because the existing one wouldn't let me reach into change some invariants for undo/redo. Which I mean, fair enough if that's what's needed, but it's a real pain to reimplement something because the API was incomplete. And of course, now if something from another library needs petgraph, I'd have to convert my graph to its graph.
So yes, in theory, if we had great APIs this wouldn't be a problem. Unfortunately, APIs are always a work in progress, and sometimes need escape hatches in order to send values between libraries.
Because no one has ever deliberately used the wrong tool for business reasons. Or thought they had a perfectly reasonable argument.
It's better to have escape hatches for in case you need them, but anyone who feels that way probably isn't using Rust to start with.
Maybe that's a bit harsh. I'm sure there are some problem domains where the other trait is desirable, but IMO it's not generic systems programming.
> Zig vs Rust also shows up with how object destruction is handled.
I often hear critiques of Drop being less efficient for anything Arena-like, where batch destruction would be better, and holding that as the reason defer is a better approach. What is not mentioned there is that there's nothing stopping you from having both. In Rust you can perform batch destruction by having additional logic (it might require control of the container and its contents' types for easiest implementation), but the default behavior remains sane.
That's fair, since you can leak the box. I will say though it's not as ergonomic as defer, as defer handles all exits from the scope, where it's trickier to juggle destructors. Though on further thought, I suppose the arena can have Drop.
EDIT: What you can't really do is this: https://github.com/smj-edison/zicl/blob/ea8b75a1284e5bd5a309...
Here I'm able to swap out std.MultiArrayList's backing to be backed by virtual memory, and correctly clean it up. I'm not sure you can really do that with Rust, barring making custom data structures for everything.
This isn't a new discussion it was there around the early rust days too.
And IMHO coherence and orphan rules have majorly contributed to the quality of the eco system.
can you elaborate on how have they contributed to the quality of the ecosystem?
there is no good way to handle colliding implementations. Both from parallel crates and due to changes over time.
Without it you can have many many additional forms of breakage. Worse you can have "new" breakage between two 3rd party crates without either of them changing due to some impl in a common ancestor changing (e.g. std) and this affecting two wild card implementations in each, now leading to an overlap.
When you have an overlap there are two options:
- fail compilation, but as mentioned this could be caused by a non breaking change in std in two in theory unrelated 3rd party dependencies
- try to choose one of the implementations. But that now gets very messy in multiple points: a) Which impl. to choose when. b) The user knowing which is chosen. c) Overlap with interactions with stuff like double dispatch, thread local variables, and in general side effects. The issues here are similar to specialization (and part why that is stuck in limbo), but a magnitude more complex as specialization is only (meant) for optimizations, while this can be deeply different behavior. Like `foo.bar()` with the same `use Bar as _;` might in one context return an `u32` and in another a `String`
In many other ecosystems it's not uncommon to run into having issues where certain libraries can't be used together at all. In rust that is close to not a thing (no_mange collisions and C dependencies are the only exception I can think of).
Similar, in my experience the likely hood of running into unintended breaking changes is lower in the rust ecosystem then e.g. python or js, that is partially due to coherence rules forcing a more clean design.
Also people are forced to have a somewhat clean dependency tree between crates in ways not all languages requires. This can help with incremental builds and compiler time, a area rust needs any help it can get. (As a side note, clean dependency structures in your modules can (sometimes) help will rust better parallelizing code gen, too.)
So overall it I think it's good.
Through it can be very annoying. And there is some potential for improvement in many ways.
---
EDIT: sorry some keyboard fat-fingering somehow submitted a half written response without me pressing enter...
EDIT 2: Fix spelling and sentence structure.
> In many other ecosystems it's not uncommon to run into having issues where certain libraries can't be used together at all.
The same problem exists in Rust, but from the other side.
If I use serde for serialization I am effectively locked in to using crates that implement serde traits (or do newtype hacks to define them myself).
If I want to use something more niche than serde, I essentially lose access to all the popular crates as they only implement serde traits.
Newtypes aren’t hacks, they’re perfectly acceptable in my opinion. Especially if you’re willing to also use a crate like `derive_more`.
In my experience using newtypes like this causes a constant shuffle between the original type and the newtype.
If a library exposes Foo and I wrap it in MyFoo implementing some trait, I need to convert to MyFoo everywhere the trait is needed and back to Foo everywhere the original type is expected.
In practice this means cluttering the code with as_foo and as_myfoo all over the place.
You could also impl From or Deref for one direction of the conversion, but it makes the code less clear in my opinion.
One strategy I like is to declare “view” types for serialization and deserialization, because you’re going to be doing that anyway if your serialized format is meant to be compatible across versions anyway.
Serde also comes with a bunch of attributes and features to make it easy to short-circuit this stuff ad hoc.
I know this only solves the serialization use case, but that seems to be where most people run into this.
honestly in my experience it rarely matters (if you care about stable APIs) as most types you want to have at an API boundary are written (or auto generated) by you
this leaves a few often small types like `DateTime<Utc>`, which you can handle with serde serialization function overwrite attributes or automatic conversions not even needing new types (through some of this attributes could be better designed)
serde is not perfect but pretty decent, but IMHO the proc macros it provides need some love/a v2 rewrite, which would only affect impl. code gen and as such is fully backward compatible, can be mixed with old code and can be from a different author (i.e. it doesn't have the problem)
Anyway that doesn't make the problem go away, just serialization/serde is both the best and worst example. (Best as it's extremely wide spread, "good enough" but not perfect, which is poison for ecosystem evolution, worst as serialization is enough of a special case to make it's best solution be potentially unusable to solve the generic problem (e.g. reflections)).
for me that is a completely different problem,
one you solve when initially writing code (so you can properly account for it and control it)
instead of a problem which can blow up when you update a package for a very pressing security fix
in the end it a question what is more important, stability or the option to monkey patch functionality into your dependencies without changing them
and given that you can always non-monkey patch crates (rust makes vendoring dep. relatively easy in case upstream doesn't fix things) I prefer the stability aspect (through if you do patch crates you re-introduce many of the issues in a different place, with the main difference of there being a chance to upstream you changes)
I've never once pulled in a new dependency and had the program fail to compile just by virtue of that dependency being present [because both my code and the new code both impl'd the same trait on the same type in some other code]. Because that can't happen because of coherence. (Right?)
It's so easy to forget about the problems we don't have because of the (good) choices people have made in the past.
> Because that can't happen because of coherence. (Right?)
yes
Through you still can run into it when unsafe is involved, e.g. C FFI/no_mange or ASM with non-mangled labels as they are globally unique. Through IMHO, it's not a common problem and has ways to make it very unlikely for the projects where it matters.
In the end if you pull in C-FFI code (or provide it) you do ope yourself up to C ABI specific problems.
There are virtually no incompatible dependencies.
The blog post gives an example of how the current approach makes dependencies incompatible.
> if someone publishes an alternative to serde (say, nextserde) then all crates which have added support for serde also need to add support for nextserde. Adding support for every new serialization library in existence is unrealistic
If I use serde, I cannot use a crate that only implements nextserde.
If I want to use nextserde, I lose the ability to use all the crates that only implement serde.
Let me rephrase: There are virtually no dependencies that when combined, cause a compiler error.
To me, the correct solution to the problem of being tied to one ecosystem crate for utility features like serialization or logging is reflection / comptime. The problem is not the orphan rule, it’s that Rust needs reflection a lot more than a dynamically-typed language does, and it should have been added a long time ago. (It’s in development now, but it will most likely be years before it ships in a stable version.)
"Note that nonbinary crates still obey the orphan rules."
I find it slightly humorous that this sentence contains three words which would be understood completely differently by the majority of the English-speaking population.
Was there some reason Rust felt the need to introduce new uses of words for concepts that already had commonly used words?
I’m not sure I fully understand but this seems to be the kind of problem that Ocaml functors solve. You program against an interface (signature) and you supply a concrete implementation (structure) when you want to run it. You can use different implementations in different parts of your application.
So maybe do something similar in Rust by expanding how you import and export modules?
I don't think explicit naming of impls is wise. They will regularly be TraitImpl or similar and add no real value. If you want to distinguish traits, perhaps force them to be within separate modules and use mod_a::mod_b::<Trait for Type> syntax.
> An interesting outcome of removing coherence and having trait bound parameters is that there becomes a meaningful difference between having a trait bound on an impl or on a struct:
This seems unfortunate to me.
I don't think fully-qualified paths are enough on their own. You also need some way to designate that an impl is symbolically unique and has to be referred to by path. Otherwise, you still end up with a problem where the compiler doesn't know which implementation to use unless you precisely name it.
You depend on crates A and B. A impls Foo for Bar. You pass an instance of Bar to a function that accepts `impl Foo`. You are happy. Later crate B adds an impl of Foo for Bar. Clearly _at least_ one of these must be an orphan impl, but both could be. Suddenly it's ambiguous which implementation of Foo you're talking about, so you break because B added an impl.
There are many potential problems of this flavor with letting any `impl Trait for Type` be an orphan impl and then referenced by path. What happens, for example, if an impl that was an orphan impl in one version of A becomes a coherent impl in a later version of A?
I think there has to be special syntax for named/path-referenced/symbolic impls, even if the impl does not have an identifier name, so that the compiler can know "this impl only resolves if you tell me _specifically this impl_" and the impl provider has a way to create a solid consumer contract about how to use that impl in particular.
Also, not having an identifier name would mean you can't have different impls of Foo for Bar in the same module. That's probably not a limitation anyone would care about, but it's there.
Using the mod name would give it a unique name, just implicitly through the module, so I don't see the issue, unless you wanted to allow a single module to allow multiple impls of the same item.
I also don't see an issue with having multiple impls of the same trait, as long as they don't provide duplicate items inside a module. I often do multiple impl blocks to break up larger logic and organize docs, though this is generally not for trait impls, but I don't see why it couldn't be.
Let me be clear though, I'm not saying this is the best path forward on the coherence/orphan situation necessarily, just a minor critique of the blog posts position. This is a famously tricky issue, and I suspect there is no silver bullet here. Though I have always wanted some way to add flexibility to the orphan rule.
> You depend on crates A and B. A impls Foo for Bar. You pass an instance of Bar to a function that accepts `impl Foo`. You are happy. Later crate B adds an impl of Foo for Bar. Clearly _at least_ one of these must be an orphan impl, but both could be. Suddenly it's ambiguous which implementation of Foo you're talking about, so you break because B added an impl.
I think that's fine. Same as what happens if B adds a new function with the same name as a function in A that you were using unqualified.
> What happens, for example, if an impl that was an orphan impl in one version of A becomes a coherent impl in a later version of A?
Nothing much?
I don’t think Rust needs this; Rust has done great for the last decade with the coherence rules it has. I am glad to not have to worry about this, and to not have to worry about any of the downstream problems (like linker errors) that coherence structurally eliminates.
The Rust ecosystem does a lot of what I like to call "inverse dependency injection".
If a Rust library needs support for TLS, typically that library implements a feature for each existing TLS backend, and keeps first-class integration which each one. The obvious thing would be to have a TLS Trait, and have each TLS library implement that trait (i.e.: dependency injection).
Because of to the orphan rule, such a trait would likely have to be declared in a small self-contained library, and each TLS library would implement that trait. I don't see any obvious impediment (aside from the fact that all TLS implementations would have to expose the same API and set of behaviours), but for some reason, the Rust ecosystem has taken the path of "every library has first-class integration with every possible provider".
This makes it really tricky to build libraries which rely on other libraries which rely on a TLS library, because your consumers can't easily select, for example, which TLS implementation to use. Libraries end up having lots of feature flags which just propagate feature flags to their dependencies.
The article itself covers the specific reasons that has led to that exact problem and the potential solutions available in the ecosystem with their various trade-offs.
Curious if they could just choose one and move on. It is really toxic to do this on every layer.
For example how bad would it be if reqwest only supports rustls and is able to have less traits/generics and compiles faster
That locks users into an ecosystem that may never evolve, which can be fine but doesn't really solve one of the core issues the author was describing. It forces the ecosystem to depend on the oldest and most incumbent crates, rather than newer ones which might be better in some ways.
It's due to the path of least resistance. One solution could be to have common traits versioned in the standard library.
Great write up of a problem that I'm glad Golang sidesteps
The problem with this is that it's systemic and central to Rusts trait-based ecosystem composition.
Go’s has a version but it's much smaller and more local. In Go, consumer-defined structural interfaces remove most of the pressure that causes the Rust problem in the first place which is producer led.
Sidesteps by not providing the same level of functionality.
As an analogy, it would be equivalent to say that "contrary to an airplane, a car sidesteps the problem of requiring wings".
Yes, indeed - but it doesn't fly.
coherence rules are one of those things that seem annoying until you maintain a library others depend on. orphan rules saved me from some nasty diamond dependency situations. the frustration is real when you just want to impl Display for some foreign type though — end up with a newtype wrapper every time and hate it.
Tangent to the topic: One of the great things about Go is that the Go team goal is to have a great developer experience. As a result, they try to bundle common third party libraries (mux, zap) into the standard library. For example, they offered an http server, but due to lacking features community packages offered convenience. The Go team used those libraries as a reference to what people wanted, and addrd a performant and simple http routing in the standard library[1].
From that link:
> We made these changes as part of our continuing effort to make Go a great language for building production systems. We studied many third-party web frameworks, extracted what we felt were the most used features, and integrated them into net/http. Then we validated our choices and improved our design by collaborating with the community in a GitHub discussion and a proposal issue. Adding these features to the standard library means one fewer dependency for many projects. But third-party web frameworks remain a fine choice for current users or programs with advanced routing needs.
[1]: https://go.dev/blog/routing-enhancements
Note the use case - someone wants to have the ability to replace a base-level crate such as serde.
When something near the bottom needs work, should there be a process for fixing it, which is a people problem? Or should there be a mechanism for bypassing it, which is a technical solution to a people problem? This is one of the curses of open source. The first approach means that there will be confrontations which must be resolved. The second means a proliferation of very similar packages.
This is part of the life cycle of an open source language. Early on, you don't have enough packages to get anything done, and are grateful that someone took the time to code something. Then it becomes clear that the early packages lacked something, and additional packages appear. Over time, you're drowning in cruft. In a previous posting, I mentioned ten years of getting a single standard ISO 8601 date parser adopted, instead of six packages with different bugs. Someone else went through the same exercise with Javascript.
Go tends to take the first approach, while Python takes the second. One of Go's strengths is that most of the core packages are maintained and used internally by Google. So you know they've been well-exercised.
Between Github and AI, it's all too easy to create minor variants of packages. Plus we now have package supply chain attacks. Curation has thus become more important. At this point in history, it's probably good to push towards the first approach.
It's a social problem that's created by a technical problem.
In many languages, if you want to integrate package A with package B, you can make and share a package AB, which people can reuse. That scales, and facilitates reuse, and avoids either package having to support everything.
In Rust, if the integration involves traits, integration between package A and package B must happen either in A or in B. That creates a scaling problem, and a social problem.
Other than duck-typed languages (and I count Go as basically that), which languages actually provide this feature?
AFAIK, it’s not really very common to be able to extend foreign types with new interfaces, especially not if you own neither.
C++ can technically do it using partial specialization, but it’s not exactly nice, and results in UB via ODR violation when it goes wrong (say you have two implementations of a `std::hash` specialization, etc.). And it only works for interfaces that are specifically designed to be specialized this way - not for vanilla dynamic dispatch, say.
> Other than duck-typed languages (and I count Go as basically that), which languages actually provide this feature?
There are only like 3 significant languages with trait-based generics, and both the other ones have some way of providing orphan instances (Haskell by requiring a flag, Scala by not having a coherence requirement at all and relying on you getting it right, which turns out to work out pretty well in practice).
More generally it's an extremely common problem to have in a mature language; if you don't have a proper fix for it then you tend to end up with awful hacks instead. Consider e.g. https://www.joda.org/joda-time-hibernate/ and https://github.com/FasterXML/jackson-datatype-joda , and note how they have to be essentially first party modules, and they have to use reflection-based runtime registries with all the associated problems. And I think that these issues significantly increased the pressure to import joda-time into the JVM system library, which ultimately came with significant downsides and costs, and in a "systems" language that aims to have a lean runtime this would be even worse.
Sure, the `chrono` library in Rust had essentially the same problem.
Scala is interesting. How do they resolve conflicts?
> Scala is interesting. How do they resolve conflicts?
If there are multiple possible instances you get a compilation error and have to specify one explicitly (which is always an option). So you do have the problem of upgrading a dependency and getting a compilation error for something that was previously fine, but it's not a big deal in practice - what I generally do is go back to the previous version and explicitly pass the instance that I was using, which is just an IDE key-combo, and then the upgrade will succeed. (After all, it's always possible to get a conflict because a library you use added a new method and the name conflicted with another library you were using - the way I see it this is essentially the same thing, just with the name being anonymous and the type being the part that matters)
You also theoretically have the much bigger problem of using two different hashing/sorting/etc. implementations with the same datastructure, which would be disastrous (although not an immediate memory corruption issue the way it could be in Rust). But in practice it's just not something I see happening, it would take a very contrived set of circumstances to encounter it.
Interesting, thanks for explaining.
> (although not an immediate memory corruption issue the way it could be in Rust)
Just to note, all of Rust's standard container types are designed such that they guarantee that buggy implementations of traits like `Hash` and `Ord` do not result in UB - just broken collections. :-)
C# isnt a duck type language (well, you can do that via dynamic keyword, but I don't know who would do that typically).
Most integration libraries in Nuget (aka c#'s cargo) are AB type libraries.
E.g. DI Container: Autofac Messaging Library: MediatR Integration: MediatR.Extensions.Autofac.DependencyInjection
There are many examples of popular libraries like this in that world.
C# does not support adding interfaces to foreign types. It does support extension classes to add methods and properties to a type, but nothing that adds fields or changes the list of interfaces implemented by a type. Rust supports this as well, because you can use traits this way.
Dependency injection is a popular solution for this problem, and you can do that as well in Rust. It requires (again) that the API is designed for dependency injection, and instead of interfaces and is-a relationships, you now have "factories" producing the implementation.
This is interesting but I wonder if you would accept that this also has the downside of moving at the speed of humans.
In a situation where you're building, I find the orphan rule frustrating because you can be stuck in a situation where you are unable to help yourself without forking half of the crates in the ecosystem.
Looking for improvements upstream, even with the absolute best solutions for option 1, has the fundamental downside that you can't unstick yourself.
This is also where I find it surprising that this article doesn't mention Scala at all. There are MANY UX/DX challenges with the implicit and witness system in Scala, so I would never guess suggest it directly, but never have I felt more enabled to solve my own problems in a language (and yes the absolute most complex, Haskell-in-Scala libraries can absolutely an impediment to this).
With AI this pace difference is even more noticeable.
I do think that the way that Scala approaches this by using imports historically was quite interesting. Using a use statement to bring a trait definition into scope isn't discussed in any of these proposals I think?
The problem is existentials, or rather the existence of existentials without the ability to explicitly override them. Even in Haskell, overriding typeclass instances requires turning off orphan checks, which is a rather large hammer.
So once you've identified this, now you might consider the universe of possible solutions to the problem. One of those solutions might be removing existentials from your language; think about how Scala would work if implicits were removed (I haven't used Scala 3, maybe this happened?). Another solution might be to decouple the whole concept of "existential implementations of typed extension points" from libraries (or crates, or however you compile and distribute code), and require bringing instances into scope via imports or similar.
Two things are true for sure, though: libraries already depend on the current behavior, whether that makes sense or not; and forcing users to understand coherence (which instance is used by which code) is almost always a giant impediment to getting users to like your language. Hence, "orphan rules", and why everyone hates Scala 2 implicits.
Yep, familiar with all of this.
That said, I would love to see a solution in my favorite class of solution: where library authors can use and benefit from this, but the average user doesn't have to notice.
I tend to think that the non-existential Scala system was _so close_, and that if you _slightly_ tweaked the scoping rules around it, you could have something great.
For example, if - as a user - I could use `.serialize(...)` from some library and it used _their_ scoped traits by default, but if I _explicitly_ (named) imported some trait(s) on my side, I could substitute my own, that'd work great.
You'd likely want to pair it with some way of e.g. allowing a per-crate prelude of explicit imports that you can ::* import within the crate to override many things at once, but... I think that with the right tweaks, you could say 'this library uses serde by default, but I can provide my own Serializer trait instead... and perhaps, if I turn off the serde Cargo feature, even their default scoped trait disappears'.
That was my first thought! I never had this problem with Scala (2.x for me, but I guess there's similar syntax/concepts in 3).
The article author does talk about naming trait impls and how to use them at call sites, but never seems to consider the idea that you could import a trait impl and use it everywhere within that scope, without extra onerous syntax.
Does this still solve the "HashMap" problem though? I guess it depends on when the named impl "binds". E.g. the named Hash impl would have to bind to the HashMap itself at creation, not at calls to `insert()` or `get()`. Which... seems like a reasonable thing?
> When something near the bottom needs work, should there be a process for fixing it, which is a people problem? Or should there be a mechanism for bypassing it, which is a technical solution to a people problem?
I don't think it's a people problem in the way we usually talk about the folly of creating technical solutions to people problems.
If something like serde is foundational, you simply can't radically change it without causing problems for lots and lots of people. That's a technical problem, not a people problem, even if serde needs radical change in order to evolve in the ways it needs to.
But sure, ok, let's imagine that wasn't the case. Let's say some new group of people decide that serde is lacking in some serious way, and they want to implement their changes. They can even do so without breaking compatibility with existing users of the crate. But the serde maintainers don't see the same problems; in fact, they believe that what this new group wants to do will actively cause more problems.
Neither group of people even needs to be right or wrong. Maybe both ways have pluses and minuses, and choosing just depends on what trade offs you value more. Neither group is wrong about wanting to either keep the status quo or make changes.
This is actually a technical problem: we need to find a way to allow both approaches coexist, without causing a ton of work for everyone else.
And even if we do run into situations where things need fixing, and things not getting fixed is a people problem, I'd argue for this particular sort of thing it's not only appropriate but essential that we have technical solutions to bypass the people problems. I mean, c'mon. People are people. People are going to be stubborn and not want change. Ossification is a real thing, and I think it's a rare project/organization that's able to avoid it. Sure, we could refuse to use technical workarounds when it's people we need to change, but in so many cases, that's just running up against a brick wall, over and over. Why do that to ourselves? Life is too short.
Having said that, I totally agree that there are situations where technical workarounds to people problems can be incredibly counter-productive, and cause more problems than they solve (like, "instead of expecting people to actually parent their kids, force everyone to give up their privacy for mandatory age verification; think of the children!"). But I don't think this is one of them.
Re: specialization and the comptime/reflection initiative
Since they allow observing whether a trait is implemented or not in the current crate they would probably become unsound if impls can be declared in downstream crates. They are a partial solution but also make other solutions harder to implement soundly (and viceversa)
Also it's sort of amazing how few people modify their tools and remove the objectionable bits.
For example, Java. Checked exceptions. Everyone hates checked exceptions. They're totally optional. Nothing in the JVM talks about a checked exception. Patch javac, comment out the checked exception checker, and compile. Nothing goes wrong. You can write Java and not deal with checked exceptions.
Likewise, you can modify rustc and make it not enforce the orphan rule.
Too many people treat their tools as black boxes and their warts as things they must tolerate and not things they can fix with their own two hands without anybody's permission.
As a non Rust man, how real are the problems in this article? Does it show up in real word or is it just a edge case? I only program in C17, C++ as C with classes and C#. Anyone can give me a good read what Traits even are?
> As a non Rust man, how real are the problems in this article?
Real, but of more concern to folks designing widely-used libraries than to folks using said libraries.
> Anyone can give me a good read what Traits even are?
You can think of traits as analogous to interfaces in OOP languages (i.e. pure virtual abstract classes in C++ terminology).
They just define a set of methods that types can implement to conform to the trait, and then consumers can treat implementing types as if they were the trait.
The major differences are: traits are implemented outside the actual type implementation, so arbitrary trait implementations can be added after the type has been written (this is why we need coherence), and rust uses traits as compile-time bounds for generics (templates).
Another imprecise analogy would be to see traits as operators.
You decide to define the operator "serde::serialize" for "MyType" but then your are stuck because you can't override or select different operators for "MyType" because only one can exists.
That's a regular yet not super common issue with traits (and this is not exclusive to Rust). It's quite irritating because you wouldn't expect this from languages with this degree of modularity.
Very real for library developers if the ecosystem started to slow down. Nonexistent for current consumers of libraries and application developers.
I've been worried about this before and the problem is real. I don 't know who maintains serde but if that gets hacked its gonna be an epic supply chain attack
Serde is maintained by dtolnay, who is a very influential figure in Rust mainly through his library development. Serde, syn, anyhow etc end up being pulled in as dependencies to nearly every Rust crate. If his account was compromised, the attack surface is essentially every single other Rust crate... not ideal
Take a look at https://contextgeneric.dev, it's as close as one can get to solving this issue without modifying rustc.
Highly Expressive Macros
No thanks. Most of the time you do not need macros and adding those is not free.
I am not sure that I need this. I can't remember to run this issue in the last couple of years.Isn't it the case that coherence is what makes Rust’s dependency graph sound? So, why would I want to give up that?
> Isn't it the case that coherence is what makes Rust’s dependency graph sound? So, why would I want to give up that?
Read the article that comment is on, it's all about why one would want that.
I have read it. I see only theoretical reasons not really practical ones. Maybe I do not use Rust enough to run into issues with coherent Rust.
Similar but not exactly the same as named impls, I'd really like to see a language handle this by separating implementing a trait from making a particular existing implementation the implicit default. Orphan rules can apply to the latter, but can be overriden in a local scope by any choice of implementation.
This is largely based on a paper I read a long time ago on how one might build a typeclass/trait system on top of an ML-style module system. But, I suspect such a setup can be beneficial even without the full module system.
Who knew that we'd long for the days of SFINAE?
Language changes could help for sure. There’s a library implementation we can use right now though: https://facet.rs/ Basically a derive macro for reflection. Yeah it’s one (more) trait to derive on all your types but then users can use that to do reflection or pretty printing or diffing or whatever they want.
Does Rust stumble over its own complexity?
It is fundamentally difficult to have an “ecosystem”.
Would much rather see a bunch of libraries that implement everything for a given use case like web-dev, embedded etc.
Unfortunately this is hard to do in rust because it is hard to implement the low level primitives.
Language’s goal should be to make building things easier imo. It should be simple to build a serde or a tokio.
From what I have seen in rust, people tend to over-engineer a single library to the absolute limit instead just building a bunch of libraries and moving on.
As an example, if it is easy to build a btreemap then you don’t have to have a bunch of traits from a bunch of different libraries pre-implemented on it. You can just copy it, adapt it a bit and move on.
Then you can have a complete thing that gives you everything you need to write a web server and it just works
So what I mean is, having a big library that implements the whole problem is better. Because then each part of that library is simple. Then I can copy paste some parts and change some others to create an alternative library. And it is better for the user of the thing because it is simple.
Having everything compatible with everything else and having everything implement every case means every individual part is over-complicated. So it is bad no matter how you combine it together.
It wasn’t ready then and it isn’t ready now.
It was always an agenda masquerading as a solution.
Ah this is very good, both the directionary tracking and getting rid of as much coherence as possible. Yay!
> Named Impls and Trait Bound Parameters
So they're finally rediscovering OCaml!
I never understood why Rust couldn't figure this shit out. Scala did.
> If a crate doesn’t implement serde’s traits for its types then those types can’t be used with serde as downstream crates cannot implement serde’s traits for another crate’s types.
You are allowed to do this in Scala.
> Worse yet, if someone publishes an alternative to serde (say, nextserde) then all crates which have added support for serde also need to add support for nextserde. Adding support for every new serialization library in existence is unrealistic and a lot of work for crate authors.
You can easily autoderive a new typeclass instance. With Scala 3, that would be:
> Here we have two overlapping trait impls which specify different values for the associated type Assoc. Scala catches this too.Perhaps I'm insufficiently caffeinated, but isn't the author describing the expression problem? That basically nails what type classes are for (in Scala and elsewhere), no?
Does the author expect incumbents to relax language rules that grant exorbitant privilege to incumbents? The orphan rule is up there with error handling in ways Rust is a screwed up language that appeals to people who don't know what they're missing. Other systems languages aren't weak in these ways.
Could you elaborate on that error handling part? To me, Rust is the only sane language I've worked with that has error-like propagation, in that functions must explicitly state what they can return, so that you don't get some bizarre runtime error thrown because the data was invalid 15 layers deeper
I don't know what quotemstr was specifically talking about, but here's my own take.
The ideal error handling is inferred algebraic effects like in Koka[1]. This allows you to add a call to an error-throwing function 15 layers down the stack and it's automatically propagated into the type signatures of all functions up the stack (and you can see the inferred effects with a language server or other tooling, similar to Rust's inferred types).
Consider the following Rust functions:
Now, how do you define E4, E5 and E6? The "correct" way is to use sum types, i.e., `enum E4 {E1(E1), E2(E2)}`, `enum E5 {E1(E1), E3(E3)}` and `enum E6 {E1(E1), E2(E2), E3(E3)}` with the appropriate From traits. The problem is that this involves a ton of boilerplate even with thiserror handling some stuff like the From traits.Since this is such a massive pain, Rust programs tend to instead either define a single error enum type that has all possible errors in the crate, or just use opaque errors like the anyhow crate. The downside is that these approaches lose type information: you no longer know that a function can't return some specific error (unless it returns no errors at all, which is rare), which is ultimately not so different from those languages where you have to guard against bizarre runtime errors.
Worse yet, if f1 has to be changed such that it returns 2 new errors, then you need to go through all error types in the call stack and flatten the new errors manually into E4, E5 and E6. If you don't flatten errors, then you end up rebuilding the call stack in error types, which is a whole different can of worms.
Algebraic effects just handle all of this more conveniently. That said, an effect system like Koka's isn't viable in a systems programming language like Rust, because optimizing user-defined effects is difficult. But you could have a special compiler-blessed effect for exceptions; algebraic checked exceptions, so to speak. Rust already does this with async.
[1] https://koka-lang.github.io
Most examples and presented issues would not compile or be a real issue... I stopped reading midway
[dead]
[dead]
[dead]
[dead]
I will never stop hating on the orphan rule, a perfect summary of what’s behind a lot of rust decisions. Purism and perfectionism at the cost of making a useful language, no better way to torpedo your ecosystem and make adding dependencies really annoying for no reason. Like not even a —dangerously-disable-the-orphan-rule, just no concessions here.
I think there are legitimate criticisms of Rust that fall in this category, but the orphan rule ain’t it.
In most other languages, it is simply not possible to “add” an interface to a class you don’t own. Rust let’s you do that if you own either the type or or the interface. That’s strictly more permissive than the competition.
The reasons those other languages have for not letting you add your interface to foreign types, or extend them with new members, are exactly the same reasons that Rust has the orphan rule.
The orphan rule is annoying. It's also one of the few things stopping trait impls from turning crate boundaries into a knife fight, where two deps both decide they own the same foreign type and the compiler has no sane answer once both impls are in the graph.
Rust pays for coherence up front with wrapper types and boilerplate, which is ugly, but the alternative is the kind of ambient monkeypatching that makes APIs hard to reason about once a codebase gets large. A narrow escape hatch might be worth trying, but a global 'disable it' switch sounds like repo poison.
> the compiler has no sane answer once both impls are in the graph
but the user could provide an answer.
Julia allows something called "piracy" which is spiritually similar to the orphan rule. and while it is strongly discouraged, the compiler allows it. and when "ambiguities" arise (here called "coherence") it becomes a social problem, not technical, to resolve. and usually package authors are pretty willing to cooperate to resolve it!
It's not a restriction born out of purity, notably uncompromising Haskell allows orphan instances.
Good for better, better for us. Rust is choke full of hard compromises and reactionary subcultures. Just recalling ? alone.
Go: error handling stinks. Generics would be dope.
Rust: if you spent 3 weeks understanding the syntax and borrow-checker, here are all of the other problems, and the list keeps growing.
Man this cracks me up.
If you think Rust has problems, it is that you've have not understood well Rust.
Brilliant