I do not work in a functional language, but these ideas have helped me a lot anyway. The only idea here that I find less directly applicable outside purely functional languages is the "Errors as values [instead of exceptions]" one.
On the surface, it makes complete sense, the non-locality of exceptions make them hard to reason about for the same reasons that GOTO is hard to reason about, and representing failure modes by values completely eliminates this non-locality. And in purely functional languages, that's the end of the story. But in imperative languages, we can do something like this:
def my_effectful_function():
if the_thing_is_bad:
# do the failure thing
raise Exception
# or
return Failure()
return Success()
and a client of this function might do something like this:
and completely ignore the failure case. Now, ignoring the failure is possible with both the exception and the failure value, but in the case of the failure value, it's much more likely to go unnoticed. The exception version is much more in line with the "let it crash" philosophy of Erlang and Elixir, and I'm not sure if the benefits of locality outweigh those of the "let it crash" philosophy.
Have any imperative folks here successfully used the "errors as values" idea?
Rust does "errors as values" pretty well, see [1]. You can manually handle them if you want, or just apply the '?' operator to auto-propagate them out of the function. In both cases, it is obvious that there was an error handling needed.
Non-locality of exceptions is a feature, not a bug. It's so you can focus on the success case, instead of error case, when reading your code. It's usually, but not always, what you want. "errors as values" is effectively the same thing as exceptions anyway, except it's explicit - meaning it's hurting readability by default and adding extra work[0]; modern languages go to extreme length to try and paper it over with syntactic magic.
Unfortunately, the whole "exceptions vs. expected" issue is fundamentally unsolvable for as long as we stick to working on common single source of truth plaintext code. Explicit and implicit error handling are both useful in different circumstances - it's entirely a presentation issue (i.e. how code looks to you); our current paradigm forces us to precommit to one or the other, which just plain sucks.
--
[0] - See how it works in C++, where you can't easily hide the mechanism.
In general, I find that explicit code is more easily read than implicit code. I prefer static over dynamic typing, I actually _like_ the explicitness of async/await or the IO monad. If something allows me to find out information about my current context without having to move up or down the stack and reading the code in other functions, I'm pretty happy about that something, because reading code is slow and tedious. What is it about implicit code that makes you feel it's more readable?
> What is it about implicit code that makes you feel it's more readable?
It's more readable when I don't care about the implicit parts at the moment. It's less readable when I do. The key thing is, whether or not I care changes from task to task, or even within the task, possibly many times per day.
The problem with our current paradigm is that we want to work on a shared plaintext artifact (codebase), and we want it to simultaneously:
1) Express everything there is about the program;
2) Be easy to read and understand and modify by humans;
3) Be the same for everyone at all times - a shared single source of truth.
"Success path" logic, error handling, logging, async/await, authentication, etc. are all cross-cutting concerns. Now, 1) means we're forced to include all of them in code simultaneously, but this goes against 2). Like, when I'm trying to understand the overall logic of some business process, then error handling and async/await are irrelevant. They're pure noise. Yet 1) forces me to look and think about them at all times.
So the issue is, 2) is best achieved when you can "filter out" concerns you don't care about at a given moment, and operate on such simplified view of code. But 1) and 3) requires us to spell the all out, everywhere, at all times. The way this is mitigated today, is through ever more complex syntax and advanced mathematical trickery. Like your async/await keywords, or the IO monad. They're ways of compressing some concerns (or classes of concerns) into terse notation, but that comes at the cost of increased complexity and mental demand, too (I mean, explain to me what a monad is, again? :)). I believe that at this point, all modern languages are hitting the Pareto frontier of readability, so whether you use $whatever-routines instead of async, or result types instead of exceptions, it's all just making some cases more readable, at the expense of other cases.
In the same category of problems is also another "holy war": lots of small functions, vs. fewer big ones. There is no right answer here, because it depends on what your goal as a reader is at the moment - e.g. understanding the idea expressed by some logic may benefit from small functions, but debugging it often benefits from the opposite. This is, again, a faux problem, created by our tooling and insistence on the "shared plaintext single source of truth" paradigm.
Compare with code folding in IDEs. It's a view feature that lets you hide blocks of code - like loop bodies, classes, or function definitions - that you don't care about at the moment. Now imagine a similar feature existed, that would let you "fold away" error handling entirely. Or "fold away" the try/catch blocks, or all the mess of dealing with Result types. Or fold away logging. Or async/await. This is the solution we need - the ability to view the shared code through various lenses. Sacrificing 3) lets us get both 1) and 2) at the same time.
It's way better than what we do now, which is precommitting to relative importance of various cross-cutting concerns, by encoding them in how easy or hard they're to write in a given language.
The way Rust solves this is that the type Result is marked with #[must_use] - ignoring it and not deciding what to do with the error raises an warning.
Plus, if you want to bubble the error, you must intentionally use the ? operator, so you are forced to acknowledge the function you called actually may raise an error (as opposed to calling an API you're unfamiliar with and forgetting to check whether it can raise an error, and the compiler not having your back)
The way to do functional programming in imperative languages is to handle the side-effects as high up in the call-chain as possible. That would mean that you return an instance of Error from lower-level and decide in some higher caller what to do about it.
That as an alternative to throwing the error. This way you get the benefit of being able to follow the flow of control from each called function back to each caller, as opposed to control jumping around wildly because of thrown errors.
In a statically typed imperative language that would need support for sum-types, to be able to return either an error or a non-error-value. Then you would be less likely to ignore the errors by accident because you would always see the return value is maybe an error.
Isn't this also basically how Haskell does it, handling side-effectful values as high up as as possible which in Haskell means moving them into the runtime system above all user-code?
Right, I understand. But my question is, how do you _ensure_ a failure value is dealt with by clients? In purely functional languages, your clients have no choice, they'll have to do something with it. In imperative languages, they can just ignore it.
In Rust, there's a `#[must_use]` attribute that can be applied to types, such as Result, and on functions. This triggers if the return value is not used. It's only a warning though, but you could imagine a hypothetical imperative language making this a hard error
#![deny(must_use)] on the root of your crate makes it a hard error for your whole crate.
Typically what happens is that having this set on is very annoying while developing code, because we often want to test incomplete code without proper error handling before we finish it. So sometimes people will block on this kind of issue in CI, but not error out during development (a warning is more than enough)
Failing to detect a result as error-value is a good failure case to be aware of. But I think if you throw an error it is also possible for a client to fail to handle it properly. No Silver Bullet.
C# nullable reference types with our parameters can kinda resemble forcing you to at least bind variables to the outputs but there’s nothing really like the linear types that we want and deserve
Simple, your compiler/type checker/static analyzer/whatever needs to ensure that any value that is not unused gives an error. But you need some sort of static analysis for that. Without type checking you don't do that of course.
These are great ideas and patterns even if you’re not doing functional programming.
FP-first/only languages tend to push you in these directions because it makes programming with them easier.
In languages where FP is optional, it takes discipline and sometimes charisma to follow these affirmations/patterns/principles.. but they’re worth it IMO.
Maybe not in literally every language, but, to cherry pick some examples:
Java (along with many other object-oriented languages) lets you create objects that are effectively immutable by declaring all fields private and not providing any property setters or other methods that would mutate the state.
Errors as values is one of the headline features of both Go and Rust, neither of which has do notation and monads.
Functional core, imperative shell is something I first learned in C#, but I don't think it was _really_ at significantly more of a disadvantage than most other languages. The only ones that let you really enforce "functional core, imperative shell" with strong language support are the pure functional languages. But people working in, say, Ocaml or Scala somehow still survive. And they do it using the same technique that people working in Java would: code review and discipline.
None of this is to say that language-level support to make it easier to stick to these principles is not valuable. But treating it as if it were a lost cause when you don't have those features in the language is the epitome of making the perfect the enemy of the good.
Coincidentally, "functional core, imperative shell" is how most companies get to adopt F# from what I was told (and had seen). It integrates really well. C# itself is a proper multi-paradigm language nowadays, and so is also quite successful at employing functional constructs.
Depending on what functions are in there, they are. But you can make types that happen to be monads in C, too. All you need is a datatype with `return` and `bind` functions that follow a certain spec.
What makes Haskell different is that it has a language-level concept of a monad that's supported by special syntax for manipulating them. (C# does, too, for what it's worth.) Without something like that, observing that a certain type can be used as a monad is maybe more of a fun fact than anything else.
(ETA: for example, many, many languages have list types that happen to be monads. But this knowledge probably won't change anything about how you use them.)
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
I think you're confusing "make illegal states unrepresentable" with "parse, don't verify"? If your type cannot represent any invalid states, there's no way you can reach them through mutation.
The "sequences of mutations" phrasing made me think they were talking about stuff like state machines or handles for external resources—for example calling `databaseConnection.close()` on an already-closed connection, which is usually a runtime error (or maybe a no-op).
I don't see the problem with state machines. When you're dealing with something that handles external input, "the input is invalid" is, for your program, a valid state. For example a regular expression engine doesn't try to make it impossible to pass in a string that doesn't match; it just returns a result indicating that the string didn't match.
Database connections are exactly what the "functional core, imperative shell" principle is about. The idea is to handle all I/O at the boundary. So shell is never passing open database connections into the functional core; it's instead retrieving everything that's needed up front so that the core can be deterministic.
"Functional core, imperative shell" might actually be my favorite of the principles, because it makes code so much easier to test. Every time I come across a codebase whose test suite has to make intense use of mocking to cope with how they allowed concurrency to spread throughout every single layer and module in the application, I get a little bit sad that nobody did its authors the service of teaching them that you don't actually need to make your own life hard like that.
It also tends to result in less code to understand and maintain overall, IME. Because if you limit the number of places where an error is even possible, you don't get stuck having to litter your codebase with excess (and often repetitive) error handling code.
I agree with all of that, but I don't see what it has to do with this thread of the discussion.
The comment I replied to was wondering what exactly greener_grass was referring to when they said:
> outside of Functional Programming […] How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
And my guess was that they had illegal transitions between states in mind. Those are hard/impossible to statically reason about when the program is written as "sequences of mutations" (particularly when aliasing is possible).
GP seemed more worried about maintaining invariants in the face of mutability, so that's what my answer spoke to.
For modeling the data in the first place, just use the right combination of sum types, product types, and newtypes - that's not specific to functional languages. I'm sure GP knew this already without me saying it. Sum types may have been a "functional programming" thing a few decades ago but they aren't anymore.
"Make illegal states unrepresentable" can be done by encapsulating the variables inside a single data object(struct/class/module) and only exporting constraint respecting functions. Also, Algebraic Data Types can be present in FP/non-FP languages.
The Result monad can be implemented in any static language with generics (just have to write two functions) and in a dynamic language this is easy (but return will have to be like T.return as there is no implict inference).
I didn't get the relation between FCore/IShell and DSLs, the main requirement for FCore is a good immutable library. Macros help DSLs though that is orthogonal.
But really, my main point is that OOP vs FP is red herring as 3/4 aspects which characterize OOP can be potentially done in both OOP and FP, with different syntax. We shouldn't conflate the first 3 with the 4th aspect - mutability.
An OOP language with better extension mechanism for classes +immutable data structure libraries and a FP language with first class modules would converge. (ref: Racket page below and comment on Reason/OCaml down the page).
See Racket page on inter-implementability of lambda, class, on the unit(ie. a first-class module) page here (https://docs.racket-lang.org/guide/unit_versus_module.html).
Racket has first class 'class' expressions. So, a mixin is a regular function.
> How can you do "Errors as values" at a large scale without do-notation / monads?
You don't need monads for this. You just need the ability to encode your error in some term like `Either e a` and ability to eliminate those terms. In Rust for example that is the `Result` type and you use pattern matching to eliminate those terms.
That's merely syntax sugar. Haskell doesn't even have this sugar and so in a monad you have to bind explicitly, and it's fine. It's not a pyramid of doom.
Let's consider a language like C++ where it has none of those syntax sugars. The absl::StatusOr is a class with the idea that errors should be values. The standard library has std::expected since C++23. So where is your pyramid of doom?
Genuinely curious for all these follow up questions:
Is immutability exclusive to functional programming?
Is the ability to use data/values exclusive to functional programming?
Are monads exclusive to functional programming?
For discussions like this, how do we separate "it was done first in functional programming but can also be done in procedural programming" with "it cannot be followed outside of functional programming"?
> Is immutability exclusive to functional programming?
No, but immutable defaults are powerful.
E.g. in JavaScript / Python, the built-in lists and dictionarys (which are blessed with special syntax) are mutable.
> Is the ability to use data/values exclusive to functional programming?
No, but expression-orientation makes this less painful
> Are monads exclusive to functional programming?
You can hack them in by abusing co-routines or perhaps async/await in various languages, but it will never be as good as something built for this purpose.
Type-inferences, type-classes and do-notation make monads workable in practice.
You don't need coroutines or async or anything complicated to model monads, just functions and data structures. Search for "c++ monads" and you'll find a ton of examples.
As someone who's written a pure functional framework for C# [1], I'll bite...
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
Firstly, don't use mutable state, write immutable types. Secondly, write constructors that reject poorly formed data structures. Thirdly, for existing libraries with types that are mutable, create a wrapper for the library with functions that return an IO/effect monad.
> How can you do "Errors as values" at a large scale without do-notation / monads?
Luckily, from a C# PoV, we have LINQ, which is equivalent to do-notation. I agree that manual management of monadic flow would be hard without something akin to do-notation or LINQ.
You can get quite far with fluent methods, but a general monadic-bind is quite hard to chain if you want to carry all of the extracted values through to subsequent expressions (lots of nesting), so yeah, it would not be ideal in those languages. It should be stated that plenty of functional languages also don't have do-notation equivalents though.
> How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
I've never really liked the "Functional core, imperative shell" thing. I think it's an admission that you're going to give up trying to be functional when it gets difficult (i.e. interacting with the real world). It is entirely possible to be functional all the way through a code-base.
In terms of DSLs: I'm not sure I know any language that can't implement a DSL and interpreter. Most people don't realise that the Gang of Four Interpreter pattern is isomorphic to free-monads, so most imperative languages have the ability to do the equivalent of free-monads.
As the GP states, it takes discipline to stick to the constraints that a language like Haskell imposes by default. Not sure about the charisma part!
I have found that having access to a world-class compiler, tooling, and large ecosystem to be more valuable to getting shit done than the exact language you choose. So, bringing the benefits of the pure-FP world into the place where I can get shit done is better than switching to, say Haskell, where it's harder to get shit done due to ecosystem limitations.
There's also the get out of jail free card, which allows me to do some gnarly high-performance code in an imperative way. And, as long as I wrap it up in a function that acts in a referentially transparent way, then I can still compose it with the rest of my pure code without concern. I just need to be a bit more careful when I do that and make sure it's for the right reasons (i.e. later stage optimisations). That's less easy to do in FP languages.
Again, it's about discipline.
If you want to see how this can look in a mainstream, imperative-first, language. I have a few samples in the repo, the one I like to share when helping OO-peeps learn about monads and monad-transformers is this game of 21/Pontoon [2]. I suspect most people won't have seen C# look like this!
Software developed on a team where everyone has different values/principles... some times it's not the technical discipline that is required, it's convincing folks to adopt these patterns and stick with them that's the hard lift. :)
If those functions represent (by some odd coincidence) half of your code-base each (half pure, half impure). Then you still benefit from the pure functional programming half.
You can always start small and build up something that becomes progressively more stable: no code base is too imperative to benefit from some pure code. Every block of pure code, even if surrounded by impure code, is one block you don't have to worry so much about. Is it fundamentalist programming? Of course not. But slowly building out from there pays you back each time you expand the scope of the pure code.
You won't have solved all of the worlds ills, but you've made part of the world's ills better. Any pure function in an impure code-base is, by-definition: more robust, easier to compose, cacheable, parallelisable, etc. these are real benefits, doesn't matter how small you start.
So, the more fundamentalist position of "once one part of your code is impure, it all is" doesn't say anything useful. And I'm always surprised when Erik pulls that argument out, because he's usually extremely pragmatic.
In many (but not all) scenarios "Make illegal states unrepresentable" is way too expensive to implement.
Especially when dealing with a fast changing domain, having to support different versions of data shapes across long time periods: dynamic data definitions are more economic and will still provide sufficient runtime protection.
"Errors as values" - what is an error? I see this pattern misused often, because not enough thought was put into the definition of an error.
"Disk is Full" vs. "data input violates some business rule" are two very - very - different things and should be treated differently. Exceptions are the right abstraction in the first case. It's not in the second case.
""Make illegal states unrepresentable" is way too expensive to implement."
This has not been my experience. The speed increase in development not having to worry about the unrepresentable cases have been very valuable. In addition as requirements change migrating old data hasn't been a huge concern. For code changes refactoring the types helps address new cases as well.
The difficulty of Making illegal state unrepresentable depends entirely on the domain you're working on. And whether discarding the occasional invalid transaction is viable.
If you're writing a CMS/wiki software, it's gonna be pretty straightforward to do.
If you're working with transactions, trades, contracts etc, it's not.
>If you're writing a CMS/wiki software, it's gonna be pretty straightforward to do.
>If you're working with transactions, trades, contracts etc, it's not.
why not?
what's the difference between those two categories, mentioned in your last two sentences, as far as this argument about illegal states is concerned? not clear to me.
in fact, just a few weeks ago, I saw a video by yaron minsky of jane street about ocaml. the title of it might have been "why ocaml". I know he has a video by that name. just not sure whether that was the one I saw or another one by him.
it can easily be googled.
in that video, he talks a fair amount about how jane street uses ocaml, including on how they use it (including defining types for kinds of business domain data, iirc) to make "invalid states unrepresentable" - in fact, iirc, that was the title of one of the slides of his talk.
i remember that very well because i was kind of impressed by the idea, although i have not checked it out practically myself yet.
but I don't know much about this area, so I'm not saying that either he or you are wrong. just asking for clarification / explanation.
also, trades and transactions seems to be the main area that jane street works in.
> why not? what's the difference between those two categories, mentioned in your last two sentences, as far as this argument about illegal states is concerned? not clear to me.
I kinda began my comment with that reason:
The difficulty entirely depends on whether discarding the occasional invalid write is possible. If you can simply return an error and ignore the write/transaction, you're golden.
Shouldn't an unrepresentable bad state not even have been proposed as a write tho? I mean the way I understand it, if something is trying to write a bad state somewhere, it is being represented somehow isn't it?
No, because you almost never have full data autonomy in corporate contexts. And the microservice arichtecures don't make this more robust either.
I.e. a transaction will have a matching transaction in another corporations/banks system, even if you don't have a distributed monolith unlike everyone else.
I see, well, that is a good point. What is the point of any architectural consideration when you are forced to ingest garbage because others have not made any effort to properly architect their systems. Maybe a strong reason why worse is better in practice.
> If you're working with transactions, trades, contracts etc, it's not.
I don’t mean to rain on your parade here, but there’s quite a few high powered orgs in the finance world that are well known for making extensive use of functional languages.
Jane St is the most famous example but it’s not the only one. Standard Chartered Bank uses a lot of Haskell, as does Barclays and Bank of America.
For "Errors as values", I agree 100% that it's better then special values or untracked exceptions but I also think that current programming languages lack the features that allow encoding errors as values conveniently. Firstly, there is no composition of errors. If I use a library for a network call and then use another library for a database query, now the possible errors should be the union of the errors that can be returned from the either of the functions. But most practical languages lack the mechanism to do that (except OCaml). One has to define a wrapper type just to encode that particular composition. And it won't work if I want to handle for example Not Found case but not Internal Server Error. I see this is because most statically typed languages have nominal typing and not structural typing. But it is a necessity for pretty much any software otherwise people will just see that tracking errors is too much trouble in terms of composition.
> I see this is because most statically typed languages have nominal typing and not structural typing.
I don't think that's it. I think what's needed is an accumulator method for error interfaces so that you can accumulate new errors into existing errors.
> To me, this simply makes more sense: isn’t it objectively better to get a finite and predictable error value from a function than an unspecified exception that may or may not happen that you still have to guard against?
Whether an error is returned as a value or thrown is orthogonal to whether it is finite and predictable. Java has checked exceptions. In Swift you also can specify the exceptions that a function may throw. How is it any less predictable than return values?
Semantically, a thrown exception is simply a return value with debug information that gets automatically returned by the caller unless specified otherwise. It is simply a very handy way to reduce boilerplate. Isn't it objectively better to not write the same thing over and over again?
I agree with regards to checked exceptions. Unfortunately Java doesn't support any form of polymorphism over thrown exceptions so it makes your code much harder to reuse. In languages that support polymorphic effects I imagine this is less of a concern.
One way to achieve "Make illegal states unrepresentable" is by using "refined" types, a.k.a. highly constrained types. There is a "refined" library in both Haskell and Scala and the "iron" library for Scala 3.
This is a nice ideal to shoot for, but strict adherence as advocated in the article is a short path to algorithmic explosions and unusable interfaces on real life systems.
For example, if you have two options that are mutually incompatible, this principle says you don't make them booleans, but instead a strict enum type populated with only legal combinations of the options. A great idea until you have 20 options to account for and your enum is now 2^16 entries long. Then your company opens a branch in a different country with a different regulatory framework and the options list grows to 50 and you code no longer fits on a hard drive.
If you truly do have 2^16 valid and distinct behaviors, it is not possible for humans to correctly write 2^16 different code paths anyway.
More than likely, the majority of the combinations of your 29 Boolean flags are invalid. You should strive to have that checked by the program, and not only as a note in external documentation.
No one is saying int should be turned into an enum.
You only have 20 options, but making that many distinctive options is not exactly a stretch. It's not like every single set of options is its own code path, most options represents a small deviation at one particular part of the code. Most of the options aren't mutually exclusive either, only a few combinations are illegal.
Imagine a simple shipping system for example. The package might be routed via truck, boat, plane, pipeline, foot, etc... Probably even a combination of those options. The package might be low, medium, or high priority, although high priority packages are not allowed to be delivered by boat. The package might be small, large, or liquid, but liquids can't be delivered by foot. There are 195 different countries with even more regulatory regimes to consider, some of which may have different customs requirements based on mode of transport. Now imagine a problem that is actually complicated.
The idea of encoding all of this business logic into the datatype is a road to madness IMHO. Especially if the rules can change on you and require you to rework your datatypes to match. On the small scale this makes a lot of sense and is good advice, but strict adherence is impractical.
You don't need a single type to represent the entire program state.
We probably both agree that separate types for shipping methods, priorities, size, country makes sense.
The API can be designed to prevent illegal transitions between types to arrive at an invalid state. The exact implementation depends on the language.
> The idea of encoding all of this business logic into the datatype is a road to madness IMHO
The alternative is hoping that every developer remembers not to violate any of the implicit and unenforced rules.
If the system is too complicated to be represented in code, can a human realistically hold the entire state machine in their head as they make changes?
The parent is discussing a situation where you have 16 distinct, binary states many but not all of which are mutually compatible. So you can have 16 bit vector of the states, 16 binary variables or an enum of valid states - the enum would have O(2^16) members because of the distribution of valid states but unlike the others, an invalid state would not be possible to represent.
If you only have 3 states, then yes, that should be an enum, not a pair of booleans, because you have 3 states, not 2x2 independent ones. Making the 4th state unrepresentable removes code and error checking. It's also just simple domain modeling.
Your latter example needs context. In what situation have you had an enum with 2^16 states? In any case, if you generally have a reasonable number of booleans, with some states being logically invalid, then you'll need error checking on those anyway.
Leaving them as booleans gives you the ability to monkey-patch things later in an ad-hoc manner, which is useful when you're still in a prototyping phase and the requirements aren't clear (and you could argue that this is almost always the case in many problem domains; I think that would be valid criticism.) But if you've already modeled the domain and you want any assurance of correctness, then I don't see why you wouldn't enforce the constraints at the type level. Your criticism seems to me the usual trade-off between strong static typing and dynamic and/or weak monkey-typing.
Nobody says you have to have a single enum type containing all the combinations. Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them. For example if option B is only relevant when option A is set to true, you can have something like
data OptA = ATrue OptB | AFalse
data OptB = BTrue | BFalse
There are three valid combinations but no type has three alternatives. Nobody in their right mind would write out an enum with 2^16 cases. If you do, you are misusing enums and it's time to consider other tools in your language.
Nobody says you have to have a single enum type containing all the combinations.
No, no one would continue up to 2^16 and the code would get unmanageable long before that. But it's illustration of the problems starting out dealing with the invalid states of two variables using an enum because what happens when more and more variables arrive? Sure, the standard answer is "just refactor" but my experience is no client or boss wants to hear "adding this small state is going require a lot of change" and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them.
Maybe you have a good chance of combining these binary conditions in a good way. But I mention you've substituted a hard problem instance (factoring binary conditions) for an easy problem instance (checking binary conditions). Functional programming has a weird dual personality where on the one hand you hear "A functional programmer is always a smarty and solve hard problems as a matter of course" but also you hear "functional programming would be the dominant paradigm if only ... we taught people young so they wouldn't have their bad procedural habits"
> and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
But you still have to handle this in your code. Wherever you have your conditions that handle this, your nest of if statements still need to cover all of these invalid combinations and ensure your app doesn't silently do the wrong thing or just crash (better).
Changing requirements requires changing code. I don't think it's a valid argument to say "we shouldn't take that approach because as requirements change we'll have to change the code". That's essentially software development.
Practically if you don't want to use enums and want another option, use a "builder" object. Pass in all of your booleans there and have it do you validation when you call its build method. It returns a read only configuration that the rest of your system can use, and the build method fails if an invalid combination of flags are passed in.
Again you force only valid combinations to exist after you call "build". And all code relies on the config produced by the build method.
Changing requirements requires changing code. I don't think it's a valid argument to say "we shouldn't take that approach because as requirements change we'll have to change the code". That's essentially software development.
You're misunderstanding me. Of course changing requirements mean changing code. The distinction is between a situation of "a small change in a requirement means a small change in code" and "a small change in a requirement means a BIG change in code". The make "make an enum of all the legal cases" approaches produces a situation where adding more binary conditions and requirements around them resulting in each change resulting a larger increase in code. And this in turn can result in a "we have to refactor this if we get one more change" and that's even more frustrating to those dictating requirements (and not uncommon in enterprise software).
Practically if you don't want to use enums and want another option, use a "builder" object. Pass in all of your booleans there and have it do you validation when you call its build method.
Cases are fine in some circumstances, enums of legal cases are fine in other circumstances and these builder might even be useful in a few bizarre cases. The main argument I'd have is that when one passes from "this approach can be useful, let's look at the situation" to "anything without this is bad", you often wind-up with a complete misallocation of resources and a fragile inability to make changes, as can be seen in today's legacy code (which was often yesterday's "best coding practices code").
I still feel like there is an important point you're walking past.
If there are 2^16 different combinations that are relevant, then you still need to handle these 2^16 combinations in your code. If you're ready a configuration file from a user and only different subsets of them are valid, somewhere in your code you still need all of the complexity to let the user know they have passed in an invalid combination. And all of that logic is equally or more complex than an enum.
If all of those cases can be handled by a few simple "if" in your code, then you'll have only a few valid options in your enum. If you have a ton of valid options you need to list in your enum, then you'll have a tone of cases you need to handle in your code.
Your underlying complaint to me sounds like you've received a lot of complex cases via your requirements. But either way the complexity is there in your code regardless of whether you validate it upfront in an enum, or deep in our code base.
Imagine a case where you have 4 options. W, X, Y, Z.
Y and Z are mutually exclusive.
X can only be set if W is set.
If Y is set then X must be set.
Going down this road you end up encoding your business logic into your datatypes. Which is good to a degree, but makes things messy when new options are added or requirements change. Imagine a new option U is introduced that is only valid when W is unset and Z is set but allows X to be set. Your interface becomes very hard to manage with even a small amount of complexity added to the options.
This is an instance of inherent complexity. Your domain is complex. You either place them into a series of nested if statements (which is what majority of programmers do), or you place it into the type system. You cannot avoid complexity either way. We are merely arguing where this complexity belongs. Such complexity is hard to manage in either case.
What is the alternative? And is it really better than encoding into the data type?
Only option I can think of is writing unit tests for each case, which doesn't seem like too much different.
And without type encoding you never sure that invariant always hold. You can always manipulate any options in any part of the program. Then after any modification you have to perform a "sanity check" if options are in a well defined state or not. I don't see how this is better than encoding the invariant into the types.
True. And what is an "illegal state" anyway? If two options are "mutually incompatible" it means that they, as far as we know, so far, have never occured at the same time.
But that is just how things are currently. The world may change. And it's important to strive for maintainable code that can accommodate such change easily.
Expanding company operations to work in a different country is an example of this (our) "world changing". States that never occur together here, may occur together there. Or in the future more generally.
So, making illegal states non-representable is about avoiding errors in respect to the system specification. But it does not take into account the fact that in the real world specifications typically evolve and change even during development, and perhaps more so later.
So, what, you’d advocate declaring all function parameters and returns as void* to maximize flexibility with respect to future changes in requirements?
No of course not. Just that we shouldn't forget that "specs" can change and typically do. Keep an open mind.. Never say "never". Illegal states can become legal.
In this example, is it really the case that all 20 are dependent into each other so you have to combine all of them into a single enum? For the ones that are independent you can keep them as booleans and you should.
Imagine you have a program where there are 4 options, W, X, Y, Z. Y and Z can't be set at the same time, and X can't be set unless W is set. If Y is set then X must be set as well.
How do you represent this in a way that makes it impossible, even through programmer error elsewhere in the program, to have the flags in an invalid state?
You can create en enum that looks like:
enum program_state =
(
W_X_Y_NZ,
W_NX_NY_Z,
NW_NX_NY_Z,
... and so on
);
You only have these 6 options. And in this case you can easily use a SAT solver to generate the cases for you so that you don't have to write them by hand.
It is not too bad in languages with discriminated unions. It's also not hard to fake discriminated unions in languages without them, even if you will miss some of the niceties.
Rather than thinking of it as an enum, think of it as a list of contructors:
class ProgramState {
bool w, x, y, z;
ProgramState(x, z) // implies y = true, w = true
ProgramState(w, z) // cannot set x; implies y = false (cannot set y)
}
Even if the class needs all four fields, internally to represent all the possible combinations of data, there's no constructors/setters to work with them independently. (Which is also related to why "Make Impossible States Unrepresentable" also generally implies immutable, no setters at all makes it much easier to make states impossible.)
In languages with discriminated unions you might even have some natural field sharing depending on how your "constructors" are written and the memory expansion isn't necessarily "exponential".
Also to mention it, languages without discriminated unions often have generics and function types, which can be used to build discriminated unions with Church encodings:
// idiomatic typescript
type Optional<IfAbsent, IfPresent> =
| {type: 'absent', detail: IfAbsent}
| {type: 'present', value: IfPresent}
// Church-encoded version
type Either<x, y> = <z>(ifLeft: (x: x) => z, ifRight: (y: y) => z) => z
// isomorphism between the two
function church<x, y>(opt: Optional<x, y>): Either<x, y> {
return (ifLeft, ifRight) => opt.type === 'absent'? ifLeft(opt.detail) : ifRight(opt.value)
}
function unchurch<x, y>(opt: Either<x, y>): Optional<x, y> {
return opt<Optional<x,y>>(x => ({type: 'absent', detail: x}), y => ({type: 'present', value: y}))
}
In addition the Church encoding of a sum type, is a function that takes N handler functions and calls the appropriate one for the case that the data type is in. With a little squinting, this is the Visitor pattern.
interface LeftRightVisitor<X, Y, Z> {
visit(x: Left<X>): Z
visit(y: Right<Y>): Z
}
interface LeftRight<X, Y> {
accept<Z>(visitor: LeftRightVisitor<X, Y, Z>): Z;
}
class Left<X> implements LeftRight<X, any> {
constructor(public readonly x: X) {}
accept<Z>(visitor: LeftRightVisitor<X, any, Z>) {
return visitor.visit(this)
}
}
class Right<Y> implements LeftRight<any, Y> {
constructor(public readonly y: Y) {}
accept<Z>(visitor: LeftRightVisitor<any, Y, Z>) {
return visitor.visit(this)
}
}
// isomorphism
function visitify<X, Y>(opt: Optional<X, Y>): LeftRight<X, Y> {
return opt.type === 'absent' ? new Left(opt.detail) : new Right(opt.value)
}
function unvisitify<X, Y>(opt: LeftRight<X, Y>): Optional<X, Y> {
return opt.accept({
visit(value: Left<X> | Right<Y>) {
return value instanceof Left? {type: 'absent', detail: value.x} : {type: 'present', value: value.y}
}
})
}
The main difference with the usual visitor pattern is that the usual visitor pattern doesn't return anything (it expects you to be holding some mutable state and the visitor will mutate it), you can do that too if you don't have access to a suitable generic for the Z parameter.
Approximate expansion of the original claim, without direct endorsement:
Suppose you have an Order object that needs to track where some larger process is in relation to three subtasks. We could imagine say that the Inventory department needs to set a physical object aside, then the Billing department needs to successfully charge for it, then the Shipping department needs to retrieve that physical object and ship it.
You start from a description of this as "one Order contains three Subtasks" where a Subtask contains the Go-style (Optional[Result], Optional[Error]) type. This architecture almost fits into a relational database, except that foreign key constraints are a bit funky if you shove everything into one nullable Result column. But let's just have the Result be some random JSON in the Subtasks table and let our relational purists weep.
Then you read this advice and you start to see that this allows for a lot of illegal states: things could contain both a result AND an error, or neither. You eventually decide that neither, is an allowed state. These are two boolean flags representing only 3 legal states and so they need to be factored into an enum: the enum is "Pending | Success[Result] | Failure[Error]".
Well, except the problem is a bit more nuanced because the pending-states also need to be consistent among the different subtasks: there is a dependency graph among them. So you should actually have an enum that says:
See, you would have had 3x3x3 = 27 valid states before for the Order but we have reduced to only the 7 legal states. Yay!
But now consider e.g. the following mutation. On Failure cases the executives at our company mandate that we never return a failed Order to a Pending status, rather we must always create a separate Order. This Order might skip inventory and/or billing and those need to be represented separately, as Inventory Skipped[OrderID] or InventoryAndBillingSkipped[OrderID]. So now our list of states following the "no unrepresentable state" logic, should really be:
Now someone else wants to add remediation actions, but only to remediate the exact error in the failure state, so _Failure is going to mean "no remediation taken" but we need to add some _Remediation with a boolean saying whether that process has completed or not. So we add:
We're only up to 21 total states so far which is still probably manageable? But these changes do demonstrate exponential growth, which is a technical term that means that the growth of each step is some small fraction of the total growth that has happened up until that point. Because everything depends on Inventory (it's at the root of the tree), when we add a new state that the Inventory can be in (Skipped) we have to add enum cases for all of the other states, and we pay a cost proportional to the size of the tree. Similarly when everything can have an error (at the leaves of the tree), when we add a new uniform requirement for errors we have to add new leaves all throughout the tree and we pay a cost proportional to the size of the tree. (Another thing to notice about the Remediation state is that it is a Pending state for another Subtask that could have been added to the original Order whenever something moves into Failure mode.)
You get something powerful by reducing the 256ish-or-whatever states into the 21 legal states; you have a compile-time assurance that no bugs in your code have created weird states that can propagate their weirdnesses throughout the system. But you also have to maintain the 21 legal states all at once, instead of maintaining 4 subtasks each having one of 4 statuses.
This is technically correct but disingenuous. This reminds of the climate change comic where a scientist asks “What if climate change is a big hoax and we create a better world god nothing?”
This is an excellent illustration. I feel functional programming provides great tools and concept for a cohesive system (one with consistent requirements, which accomplishes a single thing, etc). But many programs you write involve parts that aren't very cohesive and trying to make them cohesive is far more work than just bolting on a case statement or similar things.
Parent currently downvoted without comments. Seems like a sad way to respond.
I feel like you might be missing the point. A case statement is not to be treated as something you bolt on as a hack. It is the right tool for the job the vast majority of the times. When you use case statements, you refine and reduce your state space. It makes code easier to understand. When you combine this with the idea of making illegal states unrepresentable, a case statement gives you an exhaustive listing of what could happen. Even a lot of Haskell programmers, after using things like the `maybe` function to eliminate Maybe types and things like `fmap` over Maybe, eventually find that using case expressions produces the clearest code even though it may be one line longer.
I really hope HN enforces a rule that a downvote must be accompanied by a reply.
FP nerd: The pure core is nice and composable, with the imperative shell at the boundary.
State Skeptic: Yes, But! How do you compose the 'pure core + impure shell' pieces?
FPN: Obviously, you compose the pure pieces separately. Your app can be built using libraries built from libraries.... And, then build the imperative shell separately.
My take is that the above solution is not so easy. (atleast to me!) (and not easy for both FP and non-FP systems).
Take an example like GUI components. Ideally, you should be able to compose several components into a single component (culminating in the app) and not have a custom implementation of a giant state store which is kept in something like Redux and you define the views and modifiers using this store.
Say, you have a bunch of UI components each given as a view computed as a function from a value and possible UI events which can either modify the value, remain unhandled or configurable as either. Ex: dialog box which handles text events but leaves the 'OK' submission to the container.
There are atleast two different kinds of composability (cue quote in SICP Ch1 by Locke) - aggregation and abstraction. Ex: Having a sequence of text inputs in the document(aggregation) and then abstracting to a list of distances between cities. This abstraction puts constraints on values of the parts, both individually(positive number) and across parts(triangle inequality). There is also extension/enrichment, the dual of abstraction.
This larger abstracted component itself is now a view dependent on a value and more abstract events. But, composing recursively leads to state being held in multiple layers and computations repeated across layers. This is somewhat ameliorated by sharing of immutable parts and react like reconciliation. But, you have to express your top->down functions incrementally which is not trivial.
FP is not a silver bullet. GUI is the classic OOP showcase.
> Ideally, you should be able compose them several of them into a single app and not have a custom implementation of a giant state
If you are suggesting that components store their state, I'm not sure about "ideal" there. That works well for small GUI applications. In GUI applications of modest size, you do want a separate, well-organized and non-redundant data layer you can make sense of, at least from my experience. Qt, specifically, allows you to do both things.
This is a digression, but regarding OOP, my somewhat provocative view, is that it is not a natural thing, but in most languages, it is atleast 4 different concepts 1. Encapsulation/Namespace, 2. Polymorphism, 3. Extensibility(Inheritance is a special case) 4.Mutability.
These four concepts are forced/complected into a 'class' construct, but they need not be.
In particular, FP only varies on 4, but languages like ML,Clojure do 1,2,3 even better than OOP languages. Modules for encapsulation, Dispatch on first or even all arguments for polymorphism and first class modules, ML style, for extensibility.
Aside: There was a recent post (https://osa1.net/posts/2024-10-09-oop-good.html) (by someone who worked on GHC no less), favorably comparing how OOP does extensibility to Haskell typeclasses, which are not first class, but modules in ML languages can do what he wants and in a much more flexible way than inheritance!
There is also the dynamic aspect of orginal OOP - message passing instead of method invocation, but this is about dynamic vs static rather than OOP vs FP.
What OOP languages have managed to do which static FP hasn't done yet is the amazing live inspectable environments which lead to iterable development like we see in Smalltalk. The challenge is to do this in a more FP way while being modular.
This page (https://reasonml.github.io/docs/en/module) is useful to see how a FP language can do what he wants. Because we have functors, which are functions from a group of modules/classes to another module/class, we can have Composition, Inheritance(single/multiple), Mixins etc.
To your main point, I wouldn't say exactly that the component stores the state. But, rather that every component provides an initial value, possible events, and a default event handler which is a function from value to value. In effect, this is partially 'storing local state', but the above pieces can be composed to create a container component.
Note that there is no option really - the app wont be reimplementing how a key is handled in a text box. But composability means that the same principle should hold not just for OS/browser components but also for higher level components (A custom map or a tree-view where there are restrictions on types and number of nodes - these should also have default handling and delegation to upper levels.)
The global store choice makes it harder to have component libraries. But, the composable alternative has its problems too - redundancy and communication which skips layers (which requires 'context' in React).
> But, composing recursively leads to state being held in multiple layers and computations repeated across layers.
True, which is why re-frame has a dependency graph and subscriptions that avoid re-computation, i.e. the data dependencies are outside any view tree.
If data changes, only active nodes (ones that have been subscribed to) will re-compute. If nothing changed in a node, any dependent nodes will not re-compute.
Doesn't skipping view layers mean that constraints held by intermediate layers can be violated?
Say a city stats(location, weather) component is held inside a region component which in turn is in charge of a product route generating component (which also contains a separate 'list of products' component).
You can't update the city coordinates safely from the top as the region component enforces that the cities are within a maximum distance from each other. The intermediate constraint would have to be lifted to the higher level and checked.
Edit: There is also a more basic problem. When your app has multiple types of data(product, city), the top level store effectively becomes a database(https://www.hytradboi.com/2022/your-frontend-needs-a-databas...). This means that for every update, you have to figure out which views change, and more specifically, which rows in a view change. This isn't trivial unless you do wholesale updates (which is slow), as effects in a database can be non-local. Your views are queries and Queries on streaming data is hard.
The whole update logic could become a core part of your system modelling which creates an NxM problem (store update, registered view -> does view update?). This function requires factoring into local functions for efficient implementation which is basically the data dependency graph.
> This is fine but it's just a rehash of old well-knowned stuff.
A significant share of the current dev community wasn't there when these principles were first described. Sometimes, bringing up old topics once again helps seeing that many people had never heard of them yet.
Do you have an example of the many books and established blogs that you can share? A newcomer might not be familiar with where to start when attempting to learn more about functional programming.
~~I can't tell if you're being sarcastic, but that's exactly what TFA does. But just to pull it out explicitly:~~
Edit: I think I misunderstood your point. You were asking for a similar kind of list as TFA from the other commenter which may not necessarily be the *same* 5 ideas
Yep, was kinda being sarcastic. IMO these sorts of posts are really valuable. They don't seem valuable when you're already familiar with the ideas and have read all the posts. But for people who are knew, they can really accelerate things.
The comment kinda reminded me of the forum comments that will answer questions with "just use Google" and another person replies "I found this thread with Google ;(".
It's hard not to giggle when the conclusion right after "Smart constructors" says "Do these ideas belong only in functional programming? While they are practiced more there...".
Ah yes, because using constructors to ensure that new objects are in a valid state is virtually unheard of in object-oriented programming.
One thing this article does is assume extreme functional mindset, I dont even think OOP enters into the authors mind- With that context, I think that statement isn't about object constructors but type constructors.
I do not work in a functional language, but these ideas have helped me a lot anyway. The only idea here that I find less directly applicable outside purely functional languages is the "Errors as values [instead of exceptions]" one.
On the surface, it makes complete sense, the non-locality of exceptions make them hard to reason about for the same reasons that GOTO is hard to reason about, and representing failure modes by values completely eliminates this non-locality. And in purely functional languages, that's the end of the story. But in imperative languages, we can do something like this:
and a client of this function might do something like this: and completely ignore the failure case. Now, ignoring the failure is possible with both the exception and the failure value, but in the case of the failure value, it's much more likely to go unnoticed. The exception version is much more in line with the "let it crash" philosophy of Erlang and Elixir, and I'm not sure if the benefits of locality outweigh those of the "let it crash" philosophy.Have any imperative folks here successfully used the "errors as values" idea?
Rust does "errors as values" pretty well, see [1]. You can manually handle them if you want, or just apply the '?' operator to auto-propagate them out of the function. In both cases, it is obvious that there was an error handling needed.
[1] https://doc.rust-lang.org/book/ch09-02-recoverable-errors-wi...
I didn't know that! Every time I hear about Rust my opinion of it grows brighter. Thanks for sharing this!
Non-locality of exceptions is a feature, not a bug. It's so you can focus on the success case, instead of error case, when reading your code. It's usually, but not always, what you want. "errors as values" is effectively the same thing as exceptions anyway, except it's explicit - meaning it's hurting readability by default and adding extra work[0]; modern languages go to extreme length to try and paper it over with syntactic magic.
Unfortunately, the whole "exceptions vs. expected" issue is fundamentally unsolvable for as long as we stick to working on common single source of truth plaintext code. Explicit and implicit error handling are both useful in different circumstances - it's entirely a presentation issue (i.e. how code looks to you); our current paradigm forces us to precommit to one or the other, which just plain sucks.
--
[0] - See how it works in C++, where you can't easily hide the mechanism.
In general, I find that explicit code is more easily read than implicit code. I prefer static over dynamic typing, I actually _like_ the explicitness of async/await or the IO monad. If something allows me to find out information about my current context without having to move up or down the stack and reading the code in other functions, I'm pretty happy about that something, because reading code is slow and tedious. What is it about implicit code that makes you feel it's more readable?
> What is it about implicit code that makes you feel it's more readable?
It's more readable when I don't care about the implicit parts at the moment. It's less readable when I do. The key thing is, whether or not I care changes from task to task, or even within the task, possibly many times per day.
The problem with our current paradigm is that we want to work on a shared plaintext artifact (codebase), and we want it to simultaneously:
1) Express everything there is about the program;
2) Be easy to read and understand and modify by humans;
3) Be the same for everyone at all times - a shared single source of truth.
"Success path" logic, error handling, logging, async/await, authentication, etc. are all cross-cutting concerns. Now, 1) means we're forced to include all of them in code simultaneously, but this goes against 2). Like, when I'm trying to understand the overall logic of some business process, then error handling and async/await are irrelevant. They're pure noise. Yet 1) forces me to look and think about them at all times.
So the issue is, 2) is best achieved when you can "filter out" concerns you don't care about at a given moment, and operate on such simplified view of code. But 1) and 3) requires us to spell the all out, everywhere, at all times. The way this is mitigated today, is through ever more complex syntax and advanced mathematical trickery. Like your async/await keywords, or the IO monad. They're ways of compressing some concerns (or classes of concerns) into terse notation, but that comes at the cost of increased complexity and mental demand, too (I mean, explain to me what a monad is, again? :)). I believe that at this point, all modern languages are hitting the Pareto frontier of readability, so whether you use $whatever-routines instead of async, or result types instead of exceptions, it's all just making some cases more readable, at the expense of other cases.
In the same category of problems is also another "holy war": lots of small functions, vs. fewer big ones. There is no right answer here, because it depends on what your goal as a reader is at the moment - e.g. understanding the idea expressed by some logic may benefit from small functions, but debugging it often benefits from the opposite. This is, again, a faux problem, created by our tooling and insistence on the "shared plaintext single source of truth" paradigm.
Compare with code folding in IDEs. It's a view feature that lets you hide blocks of code - like loop bodies, classes, or function definitions - that you don't care about at the moment. Now imagine a similar feature existed, that would let you "fold away" error handling entirely. Or "fold away" the try/catch blocks, or all the mess of dealing with Result types. Or fold away logging. Or async/await. This is the solution we need - the ability to view the shared code through various lenses. Sacrificing 3) lets us get both 1) and 2) at the same time.
It's way better than what we do now, which is precommitting to relative importance of various cross-cutting concerns, by encoding them in how easy or hard they're to write in a given language.
The way Rust solves this is that the type Result is marked with #[must_use] - ignoring it and not deciding what to do with the error raises an warning.
Plus, if you want to bubble the error, you must intentionally use the ? operator, so you are forced to acknowledge the function you called actually may raise an error (as opposed to calling an API you're unfamiliar with and forgetting to check whether it can raise an error, and the compiler not having your back)
The way to do functional programming in imperative languages is to handle the side-effects as high up in the call-chain as possible. That would mean that you return an instance of Error from lower-level and decide in some higher caller what to do about it.
That as an alternative to throwing the error. This way you get the benefit of being able to follow the flow of control from each called function back to each caller, as opposed to control jumping around wildly because of thrown errors.
In a statically typed imperative language that would need support for sum-types, to be able to return either an error or a non-error-value. Then you would be less likely to ignore the errors by accident because you would always see the return value is maybe an error.
Isn't this also basically how Haskell does it, handling side-effectful values as high up as as possible which in Haskell means moving them into the runtime system above all user-code?
Right, I understand. But my question is, how do you _ensure_ a failure value is dealt with by clients? In purely functional languages, your clients have no choice, they'll have to do something with it. In imperative languages, they can just ignore it.
In Rust, there's a `#[must_use]` attribute that can be applied to types, such as Result, and on functions. This triggers if the return value is not used. It's only a warning though, but you could imagine a hypothetical imperative language making this a hard error
#![deny(must_use)] on the root of your crate makes it a hard error for your whole crate.
Typically what happens is that having this set on is very annoying while developing code, because we often want to test incomplete code without proper error handling before we finish it. So sometimes people will block on this kind of issue in CI, but not error out during development (a warning is more than enough)
Failing to detect a result as error-value is a good failure case to be aware of. But I think if you throw an error it is also possible for a client to fail to handle it properly. No Silver Bullet.
C# nullable reference types with our parameters can kinda resemble forcing you to at least bind variables to the outputs but there’s nothing really like the linear types that we want and deserve
Simple, your compiler/type checker/static analyzer/whatever needs to ensure that any value that is not unused gives an error. But you need some sort of static analysis for that. Without type checking you don't do that of course.
These are great ideas and patterns even if you’re not doing functional programming.
FP-first/only languages tend to push you in these directions because it makes programming with them easier.
In languages where FP is optional, it takes discipline and sometimes charisma to follow these affirmations/patterns/principles.. but they’re worth it IMO.
I'm not convinced that you can follow all of these outside of Functional Programming.
How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
How can you do "Errors as values" at a large scale without do-notation / monads?
How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
Maybe not in literally every language, but, to cherry pick some examples:
Java (along with many other object-oriented languages) lets you create objects that are effectively immutable by declaring all fields private and not providing any property setters or other methods that would mutate the state.
Errors as values is one of the headline features of both Go and Rust, neither of which has do notation and monads.
Functional core, imperative shell is something I first learned in C#, but I don't think it was _really_ at significantly more of a disadvantage than most other languages. The only ones that let you really enforce "functional core, imperative shell" with strong language support are the pure functional languages. But people working in, say, Ocaml or Scala somehow still survive. And they do it using the same technique that people working in Java would: code review and discipline.
None of this is to say that language-level support to make it easier to stick to these principles is not valuable. But treating it as if it were a lost cause when you don't have those features in the language is the epitome of making the perfect the enemy of the good.
Coincidentally, "functional core, imperative shell" is how most companies get to adopt F# from what I was told (and had seen). It integrates really well. C# itself is a proper multi-paradigm language nowadays, and so is also quite successful at employing functional constructs.
I consider Rust's Result<T, E> and Option<T> to be monads. Is this incorrect?
Depending on what functions are in there, they are. But you can make types that happen to be monads in C, too. All you need is a datatype with `return` and `bind` functions that follow a certain spec.
What makes Haskell different is that it has a language-level concept of a monad that's supported by special syntax for manipulating them. (C# does, too, for what it's worth.) Without something like that, observing that a certain type can be used as a monad is maybe more of a fun fact than anything else.
(ETA: for example, many, many languages have list types that happen to be monads. But this knowledge probably won't change anything about how you use them.)
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
I think you're confusing "make illegal states unrepresentable" with "parse, don't verify"? If your type cannot represent any invalid states, there's no way you can reach them through mutation.
The "sequences of mutations" phrasing made me think they were talking about stuff like state machines or handles for external resources—for example calling `databaseConnection.close()` on an already-closed connection, which is usually a runtime error (or maybe a no-op).
I don't see the problem with state machines. When you're dealing with something that handles external input, "the input is invalid" is, for your program, a valid state. For example a regular expression engine doesn't try to make it impossible to pass in a string that doesn't match; it just returns a result indicating that the string didn't match.
Database connections are exactly what the "functional core, imperative shell" principle is about. The idea is to handle all I/O at the boundary. So shell is never passing open database connections into the functional core; it's instead retrieving everything that's needed up front so that the core can be deterministic.
"Functional core, imperative shell" might actually be my favorite of the principles, because it makes code so much easier to test. Every time I come across a codebase whose test suite has to make intense use of mocking to cope with how they allowed concurrency to spread throughout every single layer and module in the application, I get a little bit sad that nobody did its authors the service of teaching them that you don't actually need to make your own life hard like that.
It also tends to result in less code to understand and maintain overall, IME. Because if you limit the number of places where an error is even possible, you don't get stuck having to litter your codebase with excess (and often repetitive) error handling code.
I agree with all of that, but I don't see what it has to do with this thread of the discussion.
The comment I replied to was wondering what exactly greener_grass was referring to when they said:
> outside of Functional Programming […] How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
And my guess was that they had illegal transitions between states in mind. Those are hard/impossible to statically reason about when the program is written as "sequences of mutations" (particularly when aliasing is possible).
> How can you "Make illegal states unrepresentable" with mutable state
By either making the data const, or encapsulating mutable state with private fields and public methods
> How can you do "Errors as values"
Go, Rust, Odin, Zig, and many more are imperative languages that do exactly this
> How can you do "Functional core, imperative shell"
Write a function that takes the old state and returns the new state
> By either marking your data as const, or encapsulating mutations with private fields and public methods
That would seem to be making illegal states unreachable rather than unrepresentable, closer in spirit to "parse, don't verify".
GP seemed more worried about maintaining invariants in the face of mutability, so that's what my answer spoke to.
For modeling the data in the first place, just use the right combination of sum types, product types, and newtypes - that's not specific to functional languages. I'm sure GP knew this already without me saying it. Sum types may have been a "functional programming" thing a few decades ago but they aren't anymore.
"Make illegal states unrepresentable" can be done by encapsulating the variables inside a single data object(struct/class/module) and only exporting constraint respecting functions. Also, Algebraic Data Types can be present in FP/non-FP languages.
The Result monad can be implemented in any static language with generics (just have to write two functions) and in a dynamic language this is easy (but return will have to be like T.return as there is no implict inference).
I didn't get the relation between FCore/IShell and DSLs, the main requirement for FCore is a good immutable library. Macros help DSLs though that is orthogonal.
But really, my main point is that OOP vs FP is red herring as 3/4 aspects which characterize OOP can be potentially done in both OOP and FP, with different syntax. We shouldn't conflate the first 3 with the 4th aspect - mutability.
An OOP language with better extension mechanism for classes +immutable data structure libraries and a FP language with first class modules would converge. (ref: Racket page below and comment on Reason/OCaml down the page).
See Racket page on inter-implementability of lambda, class, on the unit(ie. a first-class module) page here (https://docs.racket-lang.org/guide/unit_versus_module.html). Racket has first class 'class' expressions. So, a mixin is a regular function.
> How can you do "Errors as values" at a large scale without do-notation / monads?
You don't need monads for this. You just need the ability to encode your error in some term like `Either e a` and ability to eliminate those terms. In Rust for example that is the `Result` type and you use pattern matching to eliminate those terms.
Then you end up with a pyramid of doom. Fine for small examples but it doesn't scale up easily.
Rust has special syntax (`?`) for this AFAICT.
You still don't need monads for any of this. Monads give you an ad-hoc polymorphic way of doing monadic actions.
Short circuiting on `Either` is a specific case of this. You can define your own EitherBind in any language.
Now you can bind over Either to your heart's content without needing an encoding of Monads in your language.That's merely syntax sugar. Haskell doesn't even have this sugar and so in a monad you have to bind explicitly, and it's fine. It's not a pyramid of doom.
Haskell has do-notation which is a more general kind of this syntactic sugar
Yeah that's what I said about bind.
Let's consider a language like C++ where it has none of those syntax sugars. The absl::StatusOr is a class with the idea that errors should be values. The standard library has std::expected since C++23. So where is your pyramid of doom?
`?` and `let ... else` both partially address this, yeah.
Genuinely curious for all these follow up questions:
Is immutability exclusive to functional programming?
Is the ability to use data/values exclusive to functional programming?
Are monads exclusive to functional programming?
For discussions like this, how do we separate "it was done first in functional programming but can also be done in procedural programming" with "it cannot be followed outside of functional programming"?
> Is immutability exclusive to functional programming?
No, but immutable defaults are powerful.
E.g. in JavaScript / Python, the built-in lists and dictionarys (which are blessed with special syntax) are mutable.
> Is the ability to use data/values exclusive to functional programming?
No, but expression-orientation makes this less painful
> Are monads exclusive to functional programming?
You can hack them in by abusing co-routines or perhaps async/await in various languages, but it will never be as good as something built for this purpose.
Type-inferences, type-classes and do-notation make monads workable in practice.
You don't need coroutines or async or anything complicated to model monads, just functions and data structures. Search for "c++ monads" and you'll find a ton of examples.
You need the syntax if you want it to actually work well in practice.
As someone who's written a pure functional framework for C# [1], I'll bite...
> How can you "Make illegal states unrepresentable" with mutable state and sequences of mutations that cannot be enforced with the type system?
Firstly, don't use mutable state, write immutable types. Secondly, write constructors that reject poorly formed data structures. Thirdly, for existing libraries with types that are mutable, create a wrapper for the library with functions that return an IO/effect monad.
> How can you do "Errors as values" at a large scale without do-notation / monads?
Luckily, from a C# PoV, we have LINQ, which is equivalent to do-notation. I agree that manual management of monadic flow would be hard without something akin to do-notation or LINQ.
You can get quite far with fluent methods, but a general monadic-bind is quite hard to chain if you want to carry all of the extracted values through to subsequent expressions (lots of nesting), so yeah, it would not be ideal in those languages. It should be stated that plenty of functional languages also don't have do-notation equivalents though.
> How can you do "Functional core, imperative shell" without the ability to create mini DSLs and interpreters in your language?
I've never really liked the "Functional core, imperative shell" thing. I think it's an admission that you're going to give up trying to be functional when it gets difficult (i.e. interacting with the real world). It is entirely possible to be functional all the way through a code-base.
In terms of DSLs: I'm not sure I know any language that can't implement a DSL and interpreter. Most people don't realise that the Gang of Four Interpreter pattern is isomorphic to free-monads, so most imperative languages have the ability to do the equivalent of free-monads.
As the GP states, it takes discipline to stick to the constraints that a language like Haskell imposes by default. Not sure about the charisma part!
I have found that having access to a world-class compiler, tooling, and large ecosystem to be more valuable to getting shit done than the exact language you choose. So, bringing the benefits of the pure-FP world into the place where I can get shit done is better than switching to, say Haskell, where it's harder to get shit done due to ecosystem limitations.
There's also the get out of jail free card, which allows me to do some gnarly high-performance code in an imperative way. And, as long as I wrap it up in a function that acts in a referentially transparent way, then I can still compose it with the rest of my pure code without concern. I just need to be a bit more careful when I do that and make sure it's for the right reasons (i.e. later stage optimisations). That's less easy to do in FP languages.
Again, it's about discipline.
If you want to see how this can look in a mainstream, imperative-first, language. I have a few samples in the repo, the one I like to share when helping OO-peeps learn about monads and monad-transformers is this game of 21/Pontoon [2]. I suspect most people won't have seen C# look like this!
[1] https://github.com/louthy/language-ext/
[2] https://github.com/louthy/language-ext/blob/main/Samples/Car...
> Not sure about the charisma part!
Software developed on a team where everyone has different values/principles... some times it's not the technical discipline that is required, it's convincing folks to adopt these patterns and stick with them that's the hard lift. :)
Mostly functional programming does not work (https://queue.acm.org/detail.cfm?id=2611829)
I have a lot of respect for Erik Meijer and I agree with the basic premise of the paper/article. However, I don't fully agree with Erik's position.
Let's say this was my program:
If those functions represent (by some odd coincidence) half of your code-base each (half pure, half impure). Then you still benefit from the pure functional programming half.You can always start small and build up something that becomes progressively more stable: no code base is too imperative to benefit from some pure code. Every block of pure code, even if surrounded by impure code, is one block you don't have to worry so much about. Is it fundamentalist programming? Of course not. But slowly building out from there pays you back each time you expand the scope of the pure code.
You won't have solved all of the worlds ills, but you've made part of the world's ills better. Any pure function in an impure code-base is, by-definition: more robust, easier to compose, cacheable, parallelisable, etc. these are real benefits, doesn't matter how small you start.
So, the more fundamentalist position of "once one part of your code is impure, it all is" doesn't say anything useful. And I'm always surprised when Erik pulls that argument out, because he's usually extremely pragmatic.
In many (but not all) scenarios "Make illegal states unrepresentable" is way too expensive to implement.
Especially when dealing with a fast changing domain, having to support different versions of data shapes across long time periods: dynamic data definitions are more economic and will still provide sufficient runtime protection.
"Errors as values" - what is an error? I see this pattern misused often, because not enough thought was put into the definition of an error.
"Disk is Full" vs. "data input violates some business rule" are two very - very - different things and should be treated differently. Exceptions are the right abstraction in the first case. It's not in the second case.
"Functional core, imperative shell" - 100% agreement here.
""Make illegal states unrepresentable" is way too expensive to implement."
This has not been my experience. The speed increase in development not having to worry about the unrepresentable cases have been very valuable. In addition as requirements change migrating old data hasn't been a huge concern. For code changes refactoring the types helps address new cases as well.
The difficulty of Making illegal state unrepresentable depends entirely on the domain you're working on. And whether discarding the occasional invalid transaction is viable.
If you're writing a CMS/wiki software, it's gonna be pretty straightforward to do.
If you're working with transactions, trades, contracts etc, it's not.
>If you're writing a CMS/wiki software, it's gonna be pretty straightforward to do.
>If you're working with transactions, trades, contracts etc, it's not.
why not?
what's the difference between those two categories, mentioned in your last two sentences, as far as this argument about illegal states is concerned? not clear to me.
in fact, just a few weeks ago, I saw a video by yaron minsky of jane street about ocaml. the title of it might have been "why ocaml". I know he has a video by that name. just not sure whether that was the one I saw or another one by him.
it can easily be googled.
in that video, he talks a fair amount about how jane street uses ocaml, including on how they use it (including defining types for kinds of business domain data, iirc) to make "invalid states unrepresentable" - in fact, iirc, that was the title of one of the slides of his talk.
i remember that very well because i was kind of impressed by the idea, although i have not checked it out practically myself yet.
but I don't know much about this area, so I'm not saying that either he or you are wrong. just asking for clarification / explanation.
also, trades and transactions seems to be the main area that jane street works in.
edited for spelling/wrong autocorrect:
s/one I show/one I saw/
> why not? what's the difference between those two categories, mentioned in your last two sentences, as far as this argument about illegal states is concerned? not clear to me.
I kinda began my comment with that reason: The difficulty entirely depends on whether discarding the occasional invalid write is possible. If you can simply return an error and ignore the write/transaction, you're golden.
If you can't, it becomes incredibly complicated
Shouldn't an unrepresentable bad state not even have been proposed as a write tho? I mean the way I understand it, if something is trying to write a bad state somewhere, it is being represented somehow isn't it?
No, because you almost never have full data autonomy in corporate contexts. And the microservice arichtecures don't make this more robust either.
I.e. a transaction will have a matching transaction in another corporations/banks system, even if you don't have a distributed monolith unlike everyone else.
I see, well, that is a good point. What is the point of any architectural consideration when you are forced to ingest garbage because others have not made any effort to properly architect their systems. Maybe a strong reason why worse is better in practice.
> If you're working with transactions, trades, contracts etc, it's not.
I don’t mean to rain on your parade here, but there’s quite a few high powered orgs in the finance world that are well known for making extensive use of functional languages.
Jane St is the most famous example but it’s not the only one. Standard Chartered Bank uses a lot of Haskell, as does Barclays and Bank of America.
Nobody in this thread talked about functional languages until you showed up.
For "Errors as values", I agree 100% that it's better then special values or untracked exceptions but I also think that current programming languages lack the features that allow encoding errors as values conveniently. Firstly, there is no composition of errors. If I use a library for a network call and then use another library for a database query, now the possible errors should be the union of the errors that can be returned from the either of the functions. But most practical languages lack the mechanism to do that (except OCaml). One has to define a wrapper type just to encode that particular composition. And it won't work if I want to handle for example Not Found case but not Internal Server Error. I see this is because most statically typed languages have nominal typing and not structural typing. But it is a necessity for pretty much any software otherwise people will just see that tracking errors is too much trouble in terms of composition.
> I see this is because most statically typed languages have nominal typing and not structural typing.
I don't think that's it. I think what's needed is an accumulator method for error interfaces so that you can accumulate new errors into existing errors.
But this way you don't track what you handled in type level it's only available in runtime. So you are back to untracked errors.
> Errors as values
> To me, this simply makes more sense: isn’t it objectively better to get a finite and predictable error value from a function than an unspecified exception that may or may not happen that you still have to guard against?
Whether an error is returned as a value or thrown is orthogonal to whether it is finite and predictable. Java has checked exceptions. In Swift you also can specify the exceptions that a function may throw. How is it any less predictable than return values?
Semantically, a thrown exception is simply a return value with debug information that gets automatically returned by the caller unless specified otherwise. It is simply a very handy way to reduce boilerplate. Isn't it objectively better to not write the same thing over and over again?
I agree with regards to checked exceptions. Unfortunately Java doesn't support any form of polymorphism over thrown exceptions so it makes your code much harder to reuse. In languages that support polymorphic effects I imagine this is less of a concern.
Completely agree with these.
One way to achieve "Make illegal states unrepresentable" is by using "refined" types, a.k.a. highly constrained types. There is a "refined" library in both Haskell and Scala and the "iron" library for Scala 3.
> Make illegal states unrepresentable
This is a nice ideal to shoot for, but strict adherence as advocated in the article is a short path to algorithmic explosions and unusable interfaces on real life systems.
For example, if you have two options that are mutually incompatible, this principle says you don't make them booleans, but instead a strict enum type populated with only legal combinations of the options. A great idea until you have 20 options to account for and your enum is now 2^16 entries long. Then your company opens a branch in a different country with a different regulatory framework and the options list grows to 50 and you code no longer fits on a hard drive.
The mistake here is having 2^16 valid options.
If you truly do have 2^16 valid and distinct behaviors, it is not possible for humans to correctly write 2^16 different code paths anyway.
More than likely, the majority of the combinations of your 29 Boolean flags are invalid. You should strive to have that checked by the program, and not only as a note in external documentation.
No one is saying int should be turned into an enum.
You only have 20 options, but making that many distinctive options is not exactly a stretch. It's not like every single set of options is its own code path, most options represents a small deviation at one particular part of the code. Most of the options aren't mutually exclusive either, only a few combinations are illegal.
Imagine a simple shipping system for example. The package might be routed via truck, boat, plane, pipeline, foot, etc... Probably even a combination of those options. The package might be low, medium, or high priority, although high priority packages are not allowed to be delivered by boat. The package might be small, large, or liquid, but liquids can't be delivered by foot. There are 195 different countries with even more regulatory regimes to consider, some of which may have different customs requirements based on mode of transport. Now imagine a problem that is actually complicated.
The idea of encoding all of this business logic into the datatype is a road to madness IMHO. Especially if the rules can change on you and require you to rework your datatypes to match. On the small scale this makes a lot of sense and is good advice, but strict adherence is impractical.
You don't need a single type to represent the entire program state.
We probably both agree that separate types for shipping methods, priorities, size, country makes sense.
The API can be designed to prevent illegal transitions between types to arrive at an invalid state. The exact implementation depends on the language.
> The idea of encoding all of this business logic into the datatype is a road to madness IMHO
The alternative is hoping that every developer remembers not to violate any of the implicit and unenforced rules.
If the system is too complicated to be represented in code, can a human realistically hold the entire state machine in their head as they make changes?
The parent is discussing a situation where you have 16 distinct, binary states many but not all of which are mutually compatible. So you can have 16 bit vector of the states, 16 binary variables or an enum of valid states - the enum would have O(2^16) members because of the distribution of valid states but unlike the others, an invalid state would not be possible to represent.
If you only have 3 states, then yes, that should be an enum, not a pair of booleans, because you have 3 states, not 2x2 independent ones. Making the 4th state unrepresentable removes code and error checking. It's also just simple domain modeling.
Your latter example needs context. In what situation have you had an enum with 2^16 states? In any case, if you generally have a reasonable number of booleans, with some states being logically invalid, then you'll need error checking on those anyway.
Leaving them as booleans gives you the ability to monkey-patch things later in an ad-hoc manner, which is useful when you're still in a prototyping phase and the requirements aren't clear (and you could argue that this is almost always the case in many problem domains; I think that would be valid criticism.) But if you've already modeled the domain and you want any assurance of correctness, then I don't see why you wouldn't enforce the constraints at the type level. Your criticism seems to me the usual trade-off between strong static typing and dynamic and/or weak monkey-typing.
Nobody says you have to have a single enum type containing all the combinations. Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them. For example if option B is only relevant when option A is set to true, you can have something like
There are three valid combinations but no type has three alternatives. Nobody in their right mind would write out an enum with 2^16 cases. If you do, you are misusing enums and it's time to consider other tools in your language.Nobody says you have to have a single enum type containing all the combinations.
No, no one would continue up to 2^16 and the code would get unmanageable long before that. But it's illustration of the problems starting out dealing with the invalid states of two variables using an enum because what happens when more and more variables arrive? Sure, the standard answer is "just refactor" but my experience is no client or boss wants to hear "adding this small state is going require a lot of change" and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
Chances are, you can use sum types (discriminated unions) to factor things nicely if you think about them.
Maybe you have a good chance of combining these binary conditions in a good way. But I mention you've substituted a hard problem instance (factoring binary conditions) for an easy problem instance (checking binary conditions). Functional programming has a weird dual personality where on the one hand you hear "A functional programmer is always a smarty and solve hard problems as a matter of course" but also you hear "functional programming would be the dominant paradigm if only ... we taught people young so they wouldn't have their bad procedural habits"
> and a trickle of binary conditions is a very common occurrence as is code expanding to handle these (and become excessively complicated).
But you still have to handle this in your code. Wherever you have your conditions that handle this, your nest of if statements still need to cover all of these invalid combinations and ensure your app doesn't silently do the wrong thing or just crash (better).
Changing requirements requires changing code. I don't think it's a valid argument to say "we shouldn't take that approach because as requirements change we'll have to change the code". That's essentially software development.
Practically if you don't want to use enums and want another option, use a "builder" object. Pass in all of your booleans there and have it do you validation when you call its build method. It returns a read only configuration that the rest of your system can use, and the build method fails if an invalid combination of flags are passed in.
Again you force only valid combinations to exist after you call "build". And all code relies on the config produced by the build method.
Changing requirements requires changing code. I don't think it's a valid argument to say "we shouldn't take that approach because as requirements change we'll have to change the code". That's essentially software development.
You're misunderstanding me. Of course changing requirements mean changing code. The distinction is between a situation of "a small change in a requirement means a small change in code" and "a small change in a requirement means a BIG change in code". The make "make an enum of all the legal cases" approaches produces a situation where adding more binary conditions and requirements around them resulting in each change resulting a larger increase in code. And this in turn can result in a "we have to refactor this if we get one more change" and that's even more frustrating to those dictating requirements (and not uncommon in enterprise software).
Practically if you don't want to use enums and want another option, use a "builder" object. Pass in all of your booleans there and have it do you validation when you call its build method.
Cases are fine in some circumstances, enums of legal cases are fine in other circumstances and these builder might even be useful in a few bizarre cases. The main argument I'd have is that when one passes from "this approach can be useful, let's look at the situation" to "anything without this is bad", you often wind-up with a complete misallocation of resources and a fragile inability to make changes, as can be seen in today's legacy code (which was often yesterday's "best coding practices code").
I still feel like there is an important point you're walking past.
If there are 2^16 different combinations that are relevant, then you still need to handle these 2^16 combinations in your code. If you're ready a configuration file from a user and only different subsets of them are valid, somewhere in your code you still need all of the complexity to let the user know they have passed in an invalid combination. And all of that logic is equally or more complex than an enum.
If all of those cases can be handled by a few simple "if" in your code, then you'll have only a few valid options in your enum. If you have a ton of valid options you need to list in your enum, then you'll have a tone of cases you need to handle in your code.
Your underlying complaint to me sounds like you've received a lot of complex cases via your requirements. But either way the complexity is there in your code regardless of whether you validate it upfront in an enum, or deep in our code base.
Imagine a case where you have 4 options. W, X, Y, Z.
Y and Z are mutually exclusive.
X can only be set if W is set.
If Y is set then X must be set.
Going down this road you end up encoding your business logic into your datatypes. Which is good to a degree, but makes things messy when new options are added or requirements change. Imagine a new option U is introduced that is only valid when W is unset and Z is set but allows X to be set. Your interface becomes very hard to manage with even a small amount of complexity added to the options.
This is an instance of inherent complexity. Your domain is complex. You either place them into a series of nested if statements (which is what majority of programmers do), or you place it into the type system. You cannot avoid complexity either way. We are merely arguing where this complexity belongs. Such complexity is hard to manage in either case.
What is the alternative? And is it really better than encoding into the data type?
Only option I can think of is writing unit tests for each case, which doesn't seem like too much different.
And without type encoding you never sure that invariant always hold. You can always manipulate any options in any part of the program. Then after any modification you have to perform a "sanity check" if options are in a well defined state or not. I don't see how this is better than encoding the invariant into the types.
True. And what is an "illegal state" anyway? If two options are "mutually incompatible" it means that they, as far as we know, so far, have never occured at the same time.
But that is just how things are currently. The world may change. And it's important to strive for maintainable code that can accommodate such change easily.
Expanding company operations to work in a different country is an example of this (our) "world changing". States that never occur together here, may occur together there. Or in the future more generally.
So, making illegal states non-representable is about avoiding errors in respect to the system specification. But it does not take into account the fact that in the real world specifications typically evolve and change even during development, and perhaps more so later.
So, what, you’d advocate declaring all function parameters and returns as void* to maximize flexibility with respect to future changes in requirements?
No of course not. Just that we shouldn't forget that "specs" can change and typically do. Keep an open mind.. Never say "never". Illegal states can become legal.
In this example, is it really the case that all 20 are dependent into each other so you have to combine all of them into a single enum? For the ones that are independent you can keep them as booleans and you should.
Where does this exponentional size requirement come from?
Imagine you have a program where there are 4 options, W, X, Y, Z. Y and Z can't be set at the same time, and X can't be set unless W is set. If Y is set then X must be set as well.
How do you represent this in a way that makes it impossible, even through programmer error elsewhere in the program, to have the flags in an invalid state?
You can create en enum that looks like:
Maybe I'm spoiled by TypeScript but this is how I'd do it in a structural typing system:
Yes, but certainly you can see how even a toy example with just 4 booleans is already getting complicated?
It's certainly complicated. But not WTF-level complicated.
Complexity is both subjective and relative. Allowing invalid states in your model is going to complicate things, too.
I wouldn't mind having this example in production. It will scale just fine with the number of booleans.
And the Make Impossible States Unrepresentable crowd program like that?
It is not too bad in languages with discriminated unions. It's also not hard to fake discriminated unions in languages without them, even if you will miss some of the niceties.
Rather than thinking of it as an enum, think of it as a list of contructors:
Even if the class needs all four fields, internally to represent all the possible combinations of data, there's no constructors/setters to work with them independently. (Which is also related to why "Make Impossible States Unrepresentable" also generally implies immutable, no setters at all makes it much easier to make states impossible.)In languages with discriminated unions you might even have some natural field sharing depending on how your "constructors" are written and the memory expansion isn't necessarily "exponential".
Also to mention it, languages without discriminated unions often have generics and function types, which can be used to build discriminated unions with Church encodings:
In addition the Church encoding of a sum type, is a function that takes N handler functions and calls the appropriate one for the case that the data type is in. With a little squinting, this is the Visitor pattern. The main difference with the usual visitor pattern is that the usual visitor pattern doesn't return anything (it expects you to be holding some mutable state and the visitor will mutate it), you can do that too if you don't have access to a suitable generic for the Z parameter.Approximate expansion of the original claim, without direct endorsement:
Suppose you have an Order object that needs to track where some larger process is in relation to three subtasks. We could imagine say that the Inventory department needs to set a physical object aside, then the Billing department needs to successfully charge for it, then the Shipping department needs to retrieve that physical object and ship it.
You start from a description of this as "one Order contains three Subtasks" where a Subtask contains the Go-style (Optional[Result], Optional[Error]) type. This architecture almost fits into a relational database, except that foreign key constraints are a bit funky if you shove everything into one nullable Result column. But let's just have the Result be some random JSON in the Subtasks table and let our relational purists weep.
Then you read this advice and you start to see that this allows for a lot of illegal states: things could contain both a result AND an error, or neither. You eventually decide that neither, is an allowed state. These are two boolean flags representing only 3 legal states and so they need to be factored into an enum: the enum is "Pending | Success[Result] | Failure[Error]".
Well, except the problem is a bit more nuanced because the pending-states also need to be consistent among the different subtasks: there is a dependency graph among them. So you should actually have an enum that says:
See, you would have had 3x3x3 = 27 valid states before for the Order but we have reduced to only the 7 legal states. Yay!But now consider e.g. the following mutation. On Failure cases the executives at our company mandate that we never return a failed Order to a Pending status, rather we must always create a separate Order. This Order might skip inventory and/or billing and those need to be represented separately, as Inventory Skipped[OrderID] or InventoryAndBillingSkipped[OrderID]. So now our list of states following the "no unrepresentable state" logic, should really be:
Now someone else wants to add remediation actions, but only to remediate the exact error in the failure state, so _Failure is going to mean "no remediation taken" but we need to add some _Remediation with a boolean saying whether that process has completed or not. So we add: We're only up to 21 total states so far which is still probably manageable? But these changes do demonstrate exponential growth, which is a technical term that means that the growth of each step is some small fraction of the total growth that has happened up until that point. Because everything depends on Inventory (it's at the root of the tree), when we add a new state that the Inventory can be in (Skipped) we have to add enum cases for all of the other states, and we pay a cost proportional to the size of the tree. Similarly when everything can have an error (at the leaves of the tree), when we add a new uniform requirement for errors we have to add new leaves all throughout the tree and we pay a cost proportional to the size of the tree. (Another thing to notice about the Remediation state is that it is a Pending state for another Subtask that could have been added to the original Order whenever something moves into Failure mode.)You get something powerful by reducing the 256ish-or-whatever states into the 21 legal states; you have a compile-time assurance that no bugs in your code have created weird states that can propagate their weirdnesses throughout the system. But you also have to maintain the 21 legal states all at once, instead of maintaining 4 subtasks each having one of 4 statuses.
This is technically correct but disingenuous. This reminds of the climate change comic where a scientist asks “What if climate change is a big hoax and we create a better world god nothing?”
This is an excellent illustration. I feel functional programming provides great tools and concept for a cohesive system (one with consistent requirements, which accomplishes a single thing, etc). But many programs you write involve parts that aren't very cohesive and trying to make them cohesive is far more work than just bolting on a case statement or similar things.
Parent currently downvoted without comments. Seems like a sad way to respond.
I feel like you might be missing the point. A case statement is not to be treated as something you bolt on as a hack. It is the right tool for the job the vast majority of the times. When you use case statements, you refine and reduce your state space. It makes code easier to understand. When you combine this with the idea of making illegal states unrepresentable, a case statement gives you an exhaustive listing of what could happen. Even a lot of Haskell programmers, after using things like the `maybe` function to eliminate Maybe types and things like `fmap` over Maybe, eventually find that using case expressions produces the clearest code even though it may be one line longer.
I really hope HN enforces a rule that a downvote must be accompanied by a reply.
I'm very disappointed. I was really hoping for something like the SRE affirmations - https://youtu.be/ia8Q51ouA_s
My ideal service layer has a functional core - easy to understand, easy to test.
linked from the article:
https://www.javiercasas.com/articles/functional-programming-...
FP nerd: The pure core is nice and composable, with the imperative shell at the boundary.
State Skeptic: Yes, But! How do you compose the 'pure core + impure shell' pieces?
FPN: Obviously, you compose the pure pieces separately. Your app can be built using libraries built from libraries.... And, then build the imperative shell separately.
My take is that the above solution is not so easy. (atleast to me!) (and not easy for both FP and non-FP systems).
Take an example like GUI components. Ideally, you should be able to compose several components into a single component (culminating in the app) and not have a custom implementation of a giant state store which is kept in something like Redux and you define the views and modifiers using this store.
Say, you have a bunch of UI components each given as a view computed as a function from a value and possible UI events which can either modify the value, remain unhandled or configurable as either. Ex: dialog box which handles text events but leaves the 'OK' submission to the container.
There are atleast two different kinds of composability (cue quote in SICP Ch1 by Locke) - aggregation and abstraction. Ex: Having a sequence of text inputs in the document(aggregation) and then abstracting to a list of distances between cities. This abstraction puts constraints on values of the parts, both individually(positive number) and across parts(triangle inequality). There is also extension/enrichment, the dual of abstraction.
This larger abstracted component itself is now a view dependent on a value and more abstract events. But, composing recursively leads to state being held in multiple layers and computations repeated across layers. This is somewhat ameliorated by sharing of immutable parts and react like reconciliation. But, you have to express your top->down functions incrementally which is not trivial.
FP is not a silver bullet. GUI is the classic OOP showcase.
> Ideally, you should be able compose them several of them into a single app and not have a custom implementation of a giant state
If you are suggesting that components store their state, I'm not sure about "ideal" there. That works well for small GUI applications. In GUI applications of modest size, you do want a separate, well-organized and non-redundant data layer you can make sense of, at least from my experience. Qt, specifically, allows you to do both things.
This is a digression, but regarding OOP, my somewhat provocative view, is that it is not a natural thing, but in most languages, it is atleast 4 different concepts 1. Encapsulation/Namespace, 2. Polymorphism, 3. Extensibility(Inheritance is a special case) 4.Mutability.
These four concepts are forced/complected into a 'class' construct, but they need not be.
In particular, FP only varies on 4, but languages like ML,Clojure do 1,2,3 even better than OOP languages. Modules for encapsulation, Dispatch on first or even all arguments for polymorphism and first class modules, ML style, for extensibility.
Aside: There was a recent post (https://osa1.net/posts/2024-10-09-oop-good.html) (by someone who worked on GHC no less), favorably comparing how OOP does extensibility to Haskell typeclasses, which are not first class, but modules in ML languages can do what he wants and in a much more flexible way than inheritance!
There is also the dynamic aspect of orginal OOP - message passing instead of method invocation, but this is about dynamic vs static rather than OOP vs FP.
What OOP languages have managed to do which static FP hasn't done yet is the amazing live inspectable environments which lead to iterable development like we see in Smalltalk. The challenge is to do this in a more FP way while being modular.
Interesting link, thanks.
This page (https://reasonml.github.io/docs/en/module) is useful to see how a FP language can do what he wants. Because we have functors, which are functions from a group of modules/classes to another module/class, we can have Composition, Inheritance(single/multiple), Mixins etc.
To your main point, I wouldn't say exactly that the component stores the state. But, rather that every component provides an initial value, possible events, and a default event handler which is a function from value to value. In effect, this is partially 'storing local state', but the above pieces can be composed to create a container component.
Note that there is no option really - the app wont be reimplementing how a key is handled in a text box. But composability means that the same principle should hold not just for OS/browser components but also for higher level components (A custom map or a tree-view where there are restrictions on types and number of nodes - these should also have default handling and delegation to upper levels.)
The global store choice makes it harder to have component libraries. But, the composable alternative has its problems too - redundancy and communication which skips layers (which requires 'context' in React).
> But, composing recursively leads to state being held in multiple layers and computations repeated across layers.
True, which is why re-frame has a dependency graph and subscriptions that avoid re-computation, i.e. the data dependencies are outside any view tree.
If data changes, only active nodes (ones that have been subscribed to) will re-compute. If nothing changed in a node, any dependent nodes will not re-compute.
It's a beauty.
Doesn't skipping view layers mean that constraints held by intermediate layers can be violated?
Say a city stats(location, weather) component is held inside a region component which in turn is in charge of a product route generating component (which also contains a separate 'list of products' component).
You can't update the city coordinates safely from the top as the region component enforces that the cities are within a maximum distance from each other. The intermediate constraint would have to be lifted to the higher level and checked.
Edit: There is also a more basic problem. When your app has multiple types of data(product, city), the top level store effectively becomes a database(https://www.hytradboi.com/2022/your-frontend-needs-a-databas...). This means that for every update, you have to figure out which views change, and more specifically, which rows in a view change. This isn't trivial unless you do wholesale updates (which is slow), as effects in a database can be non-local. Your views are queries and Queries on streaming data is hard.
The whole update logic could become a core part of your system modelling which creates an NxM problem (store update, registered view -> does view update?). This function requires factoring into local functions for efficient implementation which is basically the data dependency graph.
A class is just a function in the category of Javascript.
This is fine but it's just a rehash of old well-knowned stuff.
I don't see the value of learning this stuff one random blog post at a time.
There are many books and established blogs with long series of material to give you an education.
> This is fine but it's just a rehash of old well-knowned stuff.
A significant share of the current dev community wasn't there when these principles were first described. Sometimes, bringing up old topics once again helps seeing that many people had never heard of them yet.
Known by you, maybe.
None of these "affirmations" are common in any code base I work with, unfortunately.
Do you have an example of the many books and established blogs that you can share? A newcomer might not be familiar with where to start when attempting to learn more about functional programming.
I'm a big fan of Haskell Programming from First Principles. That's where more advanced ideas like Monads started clicking.
https://haskellbook.com/
Let's keep repeating it until it gets more adoption.
It's a pretty good collection and summary of ideas I've only seen scattered around, before.
Could you list the top 5 ideas, with a short summary and a link to a well-regarded blog post that goes into more detail for each?
~~I can't tell if you're being sarcastic, but that's exactly what TFA does. But just to pull it out explicitly:~~
Edit: I think I misunderstood your point. You were asking for a similar kind of list as TFA from the other commenter which may not necessarily be the *same* 5 ideas
Leaving the rest here in case it helps someone:
1. Parse, don’t validate
https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...
2. Make illegal states unrepresentable
https://fsharpforfunandprofit.com/posts/designing-with-types...
3. Errors as values
https://jessewarden.com/2021/04/errors-as-values.html
4. Functional core, imperative shell
https://www.javiercasas.com/articles/functional-programming-...
5. Smart constructor
https://wiki.haskell.org/index.php?title=Smart_constructors
Yep, was kinda being sarcastic. IMO these sorts of posts are really valuable. They don't seem valuable when you're already familiar with the ideas and have read all the posts. But for people who are knew, they can really accelerate things.
The comment kinda reminded me of the forum comments that will answer questions with "just use Google" and another person replies "I found this thread with Google ;(".
It's hard not to giggle when the conclusion right after "Smart constructors" says "Do these ideas belong only in functional programming? While they are practiced more there...".
Ah yes, because using constructors to ensure that new objects are in a valid state is virtually unheard of in object-oriented programming.
One thing this article does is assume extreme functional mindset, I dont even think OOP enters into the authors mind- With that context, I think that statement isn't about object constructors but type constructors.
[dead]