I was feeling a bit like the Petunia and thought "Oh no, not again." :-) One of the annoyances of embedded programming can be having the wheel re-invented a zillion times. I was pleased to see that the author was just describing good software architecture that creates portable code on top of an environment specific library.
For doing 'bare metal' embedded work in C you need the crt0 which is the weirdly named C startup code that satisfies the assumption the C compiler made when it compiled your code. And a set of primitives to do what the i/o drivers of an operating system would have been doing for you. And voila, your C program runs on 'bare metal.'
Another good topic associated with this is setting up hooks to make STDIN and STDOUT work for your particular setup, so that when you type printf() it just automagically works.
This will also then introduce you to the concept of a basic input/output system or BIOS which exports those primitives. Then you can take that code in flash/eprom and load a binary compilation into memory and start it and now you've got a monitor or a primitive one application at a time OS like CP/M or DOS.
Its a fun road for students who really want to understand computer systems to go down.
It is a small kernel, from only a bootloader to running elf files.
It has like 10 syscalls if I remember correctly.
It is very fun, and really makes you understand the ton of legacy support still in modern x86_64 CPUs and what the os underneath is doing with privilege levels and task switching.
I even implemented a small rom for it that has an interactive ocarina from Ocarina of Time.
This is really neat. So many engineers come out of school without ever having had this sort of 'start to finish' level of hands on experience. If you ever want to do systems or systems analysis this kind of thing will really, really help.
No BIOS necessary when we're talking about bare metal systems. printf() will just resolve to a low-level UART-based routine that writes to a FIFO to be played out to the UART when it's not busy. Hell, I've seen systems that forego the FIFO and just write to the UART blocking while writing.
I hope nobody was confused into thinking I thought a BIOS was required, I was pointing out the evolution from this to a monitor. I've written some code[1] that runs on the STM32 series that uses the newlib printf(). I created the UART code [2] that is interrupt driven[3] which gives you the fun feature that you can hit ^C and have it reset the program. (useful when your code goes into an expected place :-)).
Yup, I recall Atari ST (68000) and BBC Micro (6502) having unbuffered
and interrupt access to 6402 UART - which I used to C/ASM to fire MIDI
bytes to and from.
That's awesome. Back in the day this was the strong point of eCOS which was a bare metal "platform" for running essentially one application on x86 hardware. The x86 ecosystem has gotten so complicated that being able to do this can get you better performance for an "embedded" app than running on top of Linux or another embedded OS. That translates into your appliance type device using lower cost chips which is a win. When I was playing around with eCos a lot of the digital signage market was using it.
With AMD64 style chips? Probably not. Multi-core systems really need a scheduler to get the most out of them so perhaps there are some very specific applications where that would be a win but I cannot think of anything that isn't super specific. For ARM64 chips with a small number of cores, sure that is still a very viable too for appliance type (application specific) applications.
Also regardless of what others say, you can have a go trying to feel how it was to use BASIC in 8 bit computers to do everything their hardware exposed, or even 16 bit systems like MS-DOS, but with Python.
Get a ESP32 board, and have a go at it with MicroPython or CircuitPython,
Newlib is huge and complex (even including old K&R syntax) and adapting the build process to a new system is not trivial. I spent a lot of time with it when I re-targeted chibicc and cparser to EiGen, and finally switched to PDCLib for libc and a part of uClibc for libm; see https://github.com/rochus-keller/EiGen/tree/master/ecc/lib. The result is platform independent besides esentially one file.
For a static library it does not matter whether it is huge and complex, because you will typically link into your embedded application only a small number of functions from it.
I have used a part of newlib with many different kinds of microcontrollers and its build process has always been essentially the same as a quarter of century ago, so that the script that I have written the first time, before 2000, has always worked without problems, regardless of the target CPU.
The only tricky part that I had to figure the first time was how to split the compilation of the gcc cross-compiler into a part that is built before newlib and a part that is built after newlib.
However that is not specific to newlib, but is the method that must be used when compiling a cross-gcc with any standard C library and it has been simplified over the years, so that now there is little more to it than choosing the appropriate make targets when executing the make commands.
I have never needed to change the build process of newlib for a new system, I had just needed to replace a few functions, for things like I/O peripherals or memory allocation. However, I have never used much of newlib, mostly only stdio and memory/string functions.
> it does not matter whether it is huge and complex
I was talking about the migration effort and usage complexity, not what the compiler or linker actually sees. It may well be that Newlib can be configured for every conceivable application, but it was more important to me not to have a such a behemoth and bag full of surprises in the project with preprocessor rules and dependencies that a single developer can hardly understand or keep track of. My solution is lean, complete, and works with standard-conforming compilers on each platform I need it.
The standard C library does not belong into any project, but it normally is shared together with cross-compilers, linkers and other tools by all projects that target a certain kind of hardware architecture.
So whatever preprocessor rules and dependencies may be needed to build the tool chain, they do not have any influence on the building processes for the software projects used to develop applications.
The building of the tool chain is done again only when new tool versions become available, not during the development of applications.
I assume that you have encountered problems because you have desired to build newlib with something else than gcc + binutils, with which it can be built immediately, as delivered.
Even if for some weird reason the use of gcc is avoided for the intended application, that should have not required the use of a newlib compiled with something else than gcc, as it should be linked without problems with any other ELF object files.
That project is interesting, but it just proves my point, because that is not a software project for some concrete application intended to be run on some embedded computer, but it is a tool chain, i.e. an alternative for commonly used tool chains such as gcc + binutils + newlib.
For its intended purpose, i.e. as what must be added to gcc and binutils for obtaining a complete tool chain usable for the cross-compilation and linking of executable applications for any embedded computer, newlib works fine, with minimal headaches.
If instead of using it as intended, you want to integrate it as a component in a new and different tool chain, then I completely agree with what you have found out, that it is not a good choice.
I have reacted to your first comment because that seemed to imply that newlib is not fit for its purpose of being used in embedded programming applications, which is definitely false.
You have tried to use if for something very different, and in that context you are right, but you should have explained more of that in order to avoid confusions.
Your project seems interesting, but like in most such projects you should add on the initial page some rationale for the existence of the project, i.e. which are the features where it attempts to be different from the better known alternatives based on gcc or clang.
Following the links, one eventually reaches this succinct explanation:
"The Eigen Compiler Suite is a completely self-contained collection of software development tools. It exists to be recognized and adopted as a free development toolchain which is hopefully as useful and easy to use as its source code is intended to be approachable and comprehensible for developers and students wanting to learn, maintain, and customize a complete toolchain."
This does not mention any attempts of being better than alternatives in any direction, except for being much easier to modify if someone desires to implement some kind of compiler/linker customization.
This recommends it mostly for experimental projects, not for production projects. The former are important too, but it is good to know for what it is suitable.
> it just proves my point, because that is not a software project for some concrete application intended to be run on some embedded computer
It's a compiler kit, and I also added two C compilers, and of course I needed a standard library for those. It wouldn't make sense to have a separate project just for the standard library. Anyway, Newlib was not a good match for this for the said reasons. That was my own proposition so far. My compilers are expected to also work on embedded systems, even on bare metal.
> like in most such projects you should add on the initial page some rationale for the existence of the project
Have a look at the readmes; there is one in the root and most subdirectories.
While "newlib" is an interesting idea - the approach taken here is, in many cases, the wrong one.
You see, actually, the printf() family of functions don't actually require _any_ metal, bare or otherwise, beyond the ability to print individual characters.
For this reason, a popular approach for the case of not having a full-fledged standard library is to have a fully cross-platform implementation of the family which "exposes" a symbol dependency on a character printing function, e.g.:
void putchar_(char c);
and variants of the printf functions which take the character-printing function as a runtime parameter:
int fctprintf(void (*out)(char c, void* extra_arg), void* extra_arg, const char* format, ...);
int vfctprintf(void (*out)(char c, void* extra_arg), void* extra_arg, const char* format, va_list arg);
this is the approach taken in the standalone printf implementation I maintain, originally by Marco Paland:
As replied on your other comment, when you introduce a custom printf for an embedded platform it makes more sense to just edit in support for your local I/O backend rather than having the complexity of a putch() callback function pointer.
Bare metal printf is usually faster but (surprise surprise) platform dependent.
I remember I was trying to program Atari 8-bit using C compiler, and writing directly characters to Antic memory range WITH charcode translation was 100x faster than using printf.
However I'm not sharing this code because it won't work on UART... laughs nervously
Has anybody played with newlib, but grown the complexity as the system came together?
It seems like one thing to get a bare-bones printf() working to get you started on a bit of hardware, but as the complexity of the system grows you might want to move on from (say) pushing characters out of a serial interface onto pushing them onto a bitmapped display.
Does newlib allow you to put different hooks in there as the complexity of the system increases?
Newlib provides both a standard printf, which is necessarily big, and a printf that does not support any of the floating-point format specifiers.
The latter is small enough so that I have used it in the past with various small microcontrollers, from ancient types based on PowerPC or ARM7TDMI to more recent MCUs with Cortex-M0+.
You just need to make the right configuration choice.
// QEMU UART registers - these addresses are for QEMU's 16550A UART
#define UART_BASE 0x10000000
#define UART_THR (*(volatile char *)(UART_BASE + 0x00)) // Transmit Holding Register
#define UART_RBR (*(volatile char *)(UART_BASE + 0x00)) // Receive Buffer Register
#define UART_LSR (*(volatile char *)(UART_BASE + 0x05)) // Line Status Register
This looks odd. Why are receive and transmit buffer the same and why would you use such a weird offset? Iirc RISC-V allows that, but my gut says I'd still align this to the word size.
Backwards compatibility aside, why bother implementing additional register address decoding? Since the host already doesn't need to read THR or write RBR they can be safely combined. Some UARTs call this a DATA register instead.
In school we were taught that the OS does the printf. I think the professors were just trying to generalize to not go on tangents. But, once I learned that no embedded libc variants had printf just no output path, it got a lot easier to figure out how to get it working. I wish I knew about SWO and the magic of semihosting back then. I don't think those would be hard to explain and interestingly it's one of the few things students asked about that in the field I'm also asked how to do by coworkers (the setting up _write).
> But, once I learned that no embedded libc variants had printf just no output path
Did you mean "once I learned that no, embedded libc variants have printf"?
To clarify as I had to check, embedded libc variants do indeed have some (possibly stripped-down) implementation of printf and as you say they just lack the output path (hence custom output backends like UART, etc).
I am coding RISC-V assembly (which I run on x86_64 with a mini-interpreter) but I am careful to avoid the usage of the pseudo-instructions and the registers aliases (no compressed instruction ofc). I have a little tool to generate constant loading code, one-liner (semi-colon separated instructions).
And as a pre-processor I use a simple C preprocessor (I don't want to tie the code to the pre-processor of a specific assembler): I did that for x86_64 assembly, and I could assemble with gas, nasm and fasmng(fasm2) transparently.
I don't feel comfy using duplicate instructions for a 'R'educed instruction set.
That said, I know in some cases it could increase performance since the code would use less memory (and certainly more things which I don't know because I am not into modern advanced hardware CPU micro-architecture design).
I explicitely do disable register ABI alias names, pseudo-instructions and transparent "optimizations", because I run the RISC-V binary in my own x86_64 assembly written little RISC-V machine code interpreter which does support only the core instructions (and a linux syscall translation layer). I may start to add the compressed instructions someday though.
220k just to include studio? That's insane. I have 12k and still do IO. Just without the overblown stdio and sbrk, uart_puts is enough. And only in DEBUG mode.
I thought this was going to talk about how printf is implemented. I worked with a tiny embedded processor that had 8k imem, and printf is about 100k alone. Crazy. Switched to a more basic implementation that was around 2k, and ran much,much faster. It seems printf is pretty bloated, though I guess typical people don't care.
In most C standard libraries intended for embedded applications, including newlib, there is some configuration option to provide a printf that does not support any of the floating-point format specifiers.
That is normally enough to reduce the footprint of printf by more than an order of magnitude, making it compatible with small microcontrollers.
I implemented a secure printf_s and its API is the problem. You cannot dead-code eliminate all the unused methods. And it's type unsafe. There are much better API's to implement a safe printer with all the formatting options still. format is not one of them
The only hack I could think of is having the compiler front end generate calls to different functions based on the content of the format string, similar to how some compilers replace memset with a 32-bit memset based on the type of the destination pointer
And it all falls apart as soon as a format string cannot be known at compile time.
honestly love reading about this stuff - always makes me realize how much gets glossed over in school. you think modern cpus and all the abstraction layers help or just make things messier for folks trying to learn the real basics?
I was very confused by the title, expected someone writing their own printf — i.e. the part that parses the format string, grabs varargs, converts numbers, lines up strings, etc.
I'd have called it "Bare metal puts()" or "Bare metal write()" or something along those lines instead.
(FWIW, FreeBSD's printf() is quite easy to pluck out of its surrounding libc infrastructure and adapt/customize.)
First, it is customary etiquette to indicate when linking your own code/work.
Second, that is not a POSIX compatible printf, it lacks support for '%n$' (which is used primarily for localisation). Arguably can make sense to omit for tiny embedded platforms - but then why is there FP support?
Third, cmake and build options really seem to be overkill for something like this. Copy the code into the target project, edit it. If you use your own printf, you probably need a bunch of other custom stuff anyway.
Fourth, the output callback is a reasonable idea, but somewhat self-contradictory. You're bringing in your own printf. Just adapt it to your own I/O backend, like libc has FILE*.
> You edit the source code. That's the customisation?
I meant, customization where you don't have to write the customized code yourself, just choose some build options, or at most set preprocessor variables.
> First, it is customary etiquette to indicate when linking your own code/work.
You're right, although I was only linking to the table of CMake options. And it's only partially my code, since I'm the maintainer rather than the original author
> You're bringing in your own printf. Just adapt it to your own I/O backend, like libc has FILE.
One can always do that, but - with the output callback - you can bring in an already-compiled object, which is sometimes convenient.
> If you use your own printf, you probably need a bunch of other custom stuff anyway.
My personal use case (and the reason I adopted the library) was printf deficiencies in CUDA GPU kernels. And - I really needed nothing other than printf functions. Other people just use sprintf to format output of their mostly, or wholly, self-contained functions which write output to buffers and such. Different strokes for different folks etc.
But - I will definitely check out the link.
> Second, that is not a POSIX compatible printf, it lacks support for '%n$' (which is used primarily for localisation).*
That is true. But C99 printf and C++ printf do not support that either. ATM, the aim is completing C99 printf support (when I actually work on the library, which is not that often). So, my priority would be FP denormals and binary FP (with "%a"), before other things.
> * Arguably can make sense to omit for tiny embedded platforms - but then why is there FP support?*
It's there because people wanted it / needed it; and so far, there's not been any demand for numbered position specification.
> I meant, customization where you don't have to write the customized code yourself, just choose some build options, or at most set preprocessor variables.
Honestly, if you're shying away from customising an 1-2kloc piece of code, you probably shouldn't be using a custom printf().
Case in point: function pointers are either costly or even plain unsupported on GPU architectures. I would speculate that you aren't using the callbacks there?
> * function pointers are either costly or even plain unsupported on GPU architectures.*
When you printf() from a GPU kernel, your performance is shot anyway, so performance is not a consideration. And - function pointers work, as long as they all get resolved before runtime, and you don't try to cross CPU <-> GPU boundaries.
Well, they didn't shy away from customizing it quite a bit ;)
To be clear I was trying to say it doesn't make too much sense to try to package this as an independent "easy to use" "library" with a handful of build options. Not that it's somehow not "good enough".
Put another way: a situation where you need/want a custom printf is probably a situation where a package like this doesn't exactly help you anyway and you'll need to muck with it regardless. But the code can be used. Which is exactly what the repo you linked did.
FreeBSD’s printf is my goto, too! It’s indeed enormously simple to pluck out, instantly gives you full-featured printf, and has added features such as dumping memory as hex.
Funnily enough we're not even referring to the same one, the hexdump thing is in FreeBSD's kernel printf, I was looking at the userspace one :). Haven't looked at the kernel one myself but nice to hear it's also well-engineered.
(The problem with '%D' hexdumps is that it breaks compiler format checking… and also 'D' is a length modifier for _Decimal64 starting in ISO C23… that's why our hexdump is hooked in as '%.*pHX' instead [which still gives a warning because %p is not supposed to have a precision, but at least it's not entirely broken.])
Well, I know it's pretty easy in Debian. (It's not completely pain-free if you need unpackaged third-party libraries and/or if you are cross-compiling from one uncommon architecture to another.)
> What's the state of the art for cross-compiling in $CURRENTYEAR?
Poopy garbage dog poop.
glibc is a dumpster fire of bad design. If you want to cross-compile for an arbitrarily old version of glibc then... good luck. It can be done. But it's nightmare fuel.
tbh I haven't gone too deep because glibc is such a standard :(
but I can answer with reasonable confidence "musl surely has other problems, but not this one". It's a nice, clean, simple, single set of headers and source files. Very nice.
Well there's two flavors to this. Building glibc and building a program that links against glibc. They're not entirely the same. And you'd think the latter is easier. But you'd be wrong!
It should be trivial to compile glibc with an arbitrary build system for any target Linux platform from any OS. For example if I'm on Windows and I want to build a program that targets glibc 2.23 for Ubuntu on x86_64 target that should be easy peasy. It is not.
glibc should have ONE set of .c and .h files for the entire universe. There should be a small number of #define macros that users need to specify to build whatever weird ass flavor they need. These macros should be plainly defined in a single header file that anyone can look at.
But glibc is a pile of garbage and has generated files for every damn permutation in the universe. This is not necessary. It's a function of bad design. Code should NEVER EVER EVER have a ./configure step that generates files for the local platform. EVER.
>glibc should have ONE set of .c and .h files for the entire universe. There should be a small number of #define macros that users need to specify to build whatever weird ass flavor they need. These macros should be plainly defined in a single header file that anyone can look at.
Code generation aside, this is not really a great way to do it either. The build should include target-specific files, as opposed to creating a maze of ifdefs inside the code.
> The build should include target-specific files, as opposed to creating a maze of ifdefs inside the code.
Hard disagree. What makes you think it's a maze of ifdefs?
Compiling a library should be as simple as "compile every .c/.cpp file and link them into a static/shared lib". The nightmare maze is when you don't do that and you need to painfully figure out which files you should and shouldn't include in this particular build. It's horrible.
Far far far simpler is to stick with the rule "always compile all files". It's very simple to ifdef out an entire file with a single, easily understood ifdef at the top.
I do agree you don't want the middle of a file to be a fully of 10 different ifdef cases. There's an art to whether to switch within a file or produce different files. No hard and fast rule there.
Fundamentally you put either your branching logic in the code or in the build system. And given that source files should be compatible with an unbounded array of potential build systems it is therefore superior to solve the problem once in the code itself.
I am currently trying to get the Zig glibc source/headers to compile directly via clang and trying to figure out which files to include or not include is infuriatingly unclear and difficult. So no, I strongly and passionately disagree that build-system is where this logic should occur. It's fucking awful.
You're getting downvoted but you're not wrong. Well, it's not on purpose per se. It's just really really bad design from the 80s when we didn't know better. And unfortunately its design hasn't been fixed. So we have to deal with almost 40 years of accumulated garbage.
Indeed, that why HN comments are kind of dead (AIs, zealots with one billion accounts behind VPNs, etc). Sometimes I do still try to give an honest, but unpleasant, opinion/fact though. I should stick to raw neutral communication and information publishing.
I was feeling a bit like the Petunia and thought "Oh no, not again." :-) One of the annoyances of embedded programming can be having the wheel re-invented a zillion times. I was pleased to see that the author was just describing good software architecture that creates portable code on top of an environment specific library.
For doing 'bare metal' embedded work in C you need the crt0 which is the weirdly named C startup code that satisfies the assumption the C compiler made when it compiled your code. And a set of primitives to do what the i/o drivers of an operating system would have been doing for you. And voila, your C program runs on 'bare metal.'
Another good topic associated with this is setting up hooks to make STDIN and STDOUT work for your particular setup, so that when you type printf() it just automagically works.
This will also then introduce you to the concept of a basic input/output system or BIOS which exports those primitives. Then you can take that code in flash/eprom and load a binary compilation into memory and start it and now you've got a monitor or a primitive one application at a time OS like CP/M or DOS.
Its a fun road for students who really want to understand computer systems to go down.
At my school, we did the following project : https://github.com/lse/k
It is a small kernel, from only a bootloader to running elf files.
It has like 10 syscalls if I remember correctly.
It is very fun, and really makes you understand the ton of legacy support still in modern x86_64 CPUs and what the os underneath is doing with privilege levels and task switching.
I even implemented a small rom for it that has an interactive ocarina from Ocarina of Time.
This is really neat. So many engineers come out of school without ever having had this sort of 'start to finish' level of hands on experience. If you ever want to do systems or systems analysis this kind of thing will really, really help.
Also, if anyone wants to try it, the whole course is available publicly here (in English) https://k.lse.epita.fr/
What is your school? I thought it was the London School of Economics, but it’s another LSE.
It's EPITA, in France.
LSE is the System's laboratory of EPITA (https://www.lse.epita.fr/)
No BIOS necessary when we're talking about bare metal systems. printf() will just resolve to a low-level UART-based routine that writes to a FIFO to be played out to the UART when it's not busy. Hell, I've seen systems that forego the FIFO and just write to the UART blocking while writing.
I hope nobody was confused into thinking I thought a BIOS was required, I was pointing out the evolution from this to a monitor. I've written some code[1] that runs on the STM32 series that uses the newlib printf(). I created the UART code [2] that is interrupt driven[3] which gives you the fun feature that you can hit ^C and have it reset the program. (useful when your code goes into an expected place :-)).
[1] https://github.com/ChuckM/
[2] https://github.com/ChuckM/nucleo/blob/master/f446re/uart/uar...
[3] https://github.com/ChuckM/nucleo/blob/master/f446re/common/u...
Yup, I recall Atari ST (68000) and BBC Micro (6502) having unbuffered and interrupt access to 6402 UART - which I used to C/ASM to fire MIDI bytes to and from.
This was my attempt at a minimal bare-metal C environment:
https://github.com/marssaxman/startc
That's awesome. Back in the day this was the strong point of eCOS which was a bare metal "platform" for running essentially one application on x86 hardware. The x86 ecosystem has gotten so complicated that being able to do this can get you better performance for an "embedded" app than running on top of Linux or another embedded OS. That translates into your appliance type device using lower cost chips which is a win. When I was playing around with eCos a lot of the digital signage market was using it.
Does anyone still do it that way?
With AMD64 style chips? Probably not. Multi-core systems really need a scheduler to get the most out of them so perhaps there are some very specific applications where that would be a win but I cannot think of anything that isn't super specific. For ARM64 chips with a small number of cores, sure that is still a very viable too for appliance type (application specific) applications.
This sounds fascinating and absolutely alien to me, a Python dev. Any good books or other sources to learn more you can recommend?
You can start here, https://wiki.osdev.org/Expanded_Main_Page
Also regardless of what others say, you can have a go trying to feel how it was to use BASIC in 8 bit computers to do everything their hardware exposed, or even 16 bit systems like MS-DOS, but with Python.
Get a ESP32 board, and have a go at it with MicroPython or CircuitPython,
https://docs.micropython.org/en/latest/esp32/quickref.html
https://learn.adafruit.com/circuitpython-with-esp32-quick-st...
There's always the Minix book!
Newlib is huge and complex (even including old K&R syntax) and adapting the build process to a new system is not trivial. I spent a lot of time with it when I re-targeted chibicc and cparser to EiGen, and finally switched to PDCLib for libc and a part of uClibc for libm; see https://github.com/rochus-keller/EiGen/tree/master/ecc/lib. The result is platform independent besides esentially one file.
For a static library it does not matter whether it is huge and complex, because you will typically link into your embedded application only a small number of functions from it.
I have used a part of newlib with many different kinds of microcontrollers and its build process has always been essentially the same as a quarter of century ago, so that the script that I have written the first time, before 2000, has always worked without problems, regardless of the target CPU.
The only tricky part that I had to figure the first time was how to split the compilation of the gcc cross-compiler into a part that is built before newlib and a part that is built after newlib.
However that is not specific to newlib, but is the method that must be used when compiling a cross-gcc with any standard C library and it has been simplified over the years, so that now there is little more to it than choosing the appropriate make targets when executing the make commands.
I have never needed to change the build process of newlib for a new system, I had just needed to replace a few functions, for things like I/O peripherals or memory allocation. However, I have never used much of newlib, mostly only stdio and memory/string functions.
> it does not matter whether it is huge and complex
I was talking about the migration effort and usage complexity, not what the compiler or linker actually sees. It may well be that Newlib can be configured for every conceivable application, but it was more important to me not to have a such a behemoth and bag full of surprises in the project with preprocessor rules and dependencies that a single developer can hardly understand or keep track of. My solution is lean, complete, and works with standard-conforming compilers on each platform I need it.
The standard C library does not belong into any project, but it normally is shared together with cross-compilers, linkers and other tools by all projects that target a certain kind of hardware architecture.
So whatever preprocessor rules and dependencies may be needed to build the tool chain, they do not have any influence on the building processes for the software projects used to develop applications.
The building of the tool chain is done again only when new tool versions become available, not during the development of applications.
I assume that you have encountered problems because you have desired to build newlib with something else than gcc + binutils, with which it can be built immediately, as delivered.
Even if for some weird reason the use of gcc is avoided for the intended application, that should have not required the use of a newlib compiled with something else than gcc, as it should be linked without problems with any other ELF object files.
> The standard C library does not belong into any project
Why not? Have a look at https://github.com/rochus-keller/Eigen.
> because you have desired to build newlib with something else than gcc + binutils
Well, the whole point was to make it compatible with my own C compilers.
That project is interesting, but it just proves my point, because that is not a software project for some concrete application intended to be run on some embedded computer, but it is a tool chain, i.e. an alternative for commonly used tool chains such as gcc + binutils + newlib.
For its intended purpose, i.e. as what must be added to gcc and binutils for obtaining a complete tool chain usable for the cross-compilation and linking of executable applications for any embedded computer, newlib works fine, with minimal headaches.
If instead of using it as intended, you want to integrate it as a component in a new and different tool chain, then I completely agree with what you have found out, that it is not a good choice.
I have reacted to your first comment because that seemed to imply that newlib is not fit for its purpose of being used in embedded programming applications, which is definitely false.
You have tried to use if for something very different, and in that context you are right, but you should have explained more of that in order to avoid confusions.
Your project seems interesting, but like in most such projects you should add on the initial page some rationale for the existence of the project, i.e. which are the features where it attempts to be different from the better known alternatives based on gcc or clang.
Following the links, one eventually reaches this succinct explanation:
"The Eigen Compiler Suite is a completely self-contained collection of software development tools. It exists to be recognized and adopted as a free development toolchain which is hopefully as useful and easy to use as its source code is intended to be approachable and comprehensible for developers and students wanting to learn, maintain, and customize a complete toolchain."
This does not mention any attempts of being better than alternatives in any direction, except for being much easier to modify if someone desires to implement some kind of compiler/linker customization.
This recommends it mostly for experimental projects, not for production projects. The former are important too, but it is good to know for what it is suitable.
> it just proves my point, because that is not a software project for some concrete application intended to be run on some embedded computer
It's a compiler kit, and I also added two C compilers, and of course I needed a standard library for those. It wouldn't make sense to have a separate project just for the standard library. Anyway, Newlib was not a good match for this for the said reasons. That was my own proposition so far. My compilers are expected to also work on embedded systems, even on bare metal.
> like in most such projects you should add on the initial page some rationale for the existence of the project
Have a look at the readmes; there is one in the root and most subdirectories.
EDIT: or just ask Perplexity: https://www.perplexity.ai/search/can-you-explain-what-this-p...
While "newlib" is an interesting idea - the approach taken here is, in many cases, the wrong one.
You see, actually, the printf() family of functions don't actually require _any_ metal, bare or otherwise, beyond the ability to print individual characters.
For this reason, a popular approach for the case of not having a full-fledged standard library is to have a fully cross-platform implementation of the family which "exposes" a symbol dependency on a character printing function, e.g.:
and variants of the printf functions which take the character-printing function as a runtime parameter: this is the approach taken in the standalone printf implementation I maintain, originally by Marco Paland:https://github.com/eyalroz/printf
As replied on your other comment, when you introduce a custom printf for an embedded platform it makes more sense to just edit in support for your local I/O backend rather than having the complexity of a putch() callback function pointer.
cf. https://news.ycombinator.com/item?id=43811191 for other notes.
Bare metal printf is usually faster but (surprise surprise) platform dependent.
I remember I was trying to program Atari 8-bit using C compiler, and writing directly characters to Antic memory range WITH charcode translation was 100x faster than using printf.
However I'm not sharing this code because it won't work on UART... laughs nervously
Has anybody played with newlib, but grown the complexity as the system came together?
It seems like one thing to get a bare-bones printf() working to get you started on a bit of hardware, but as the complexity of the system grows you might want to move on from (say) pushing characters out of a serial interface onto pushing them onto a bitmapped display.
Does newlib allow you to put different hooks in there as the complexity of the system increases?
Newlib provides both a standard printf, which is necessarily big, and a printf that does not support any of the floating-point format specifiers.
The latter is small enough so that I have used it in the past with various small microcontrollers, from ancient types based on PowerPC or ARM7TDMI to more recent MCUs with Cortex-M0+.
You just need to make the right configuration choice.
You can always write a printf replacement that takes a minimal control block that provides put, get, control, and a context.
That way you can print to a serial port, an LCD Display, or a log.
Meaning seriously the standard printf is late 1970's hot garbage and no one should use it.
> Why are receive and transmit buffer the same?
Backwards compatibility aside, why bother implementing additional register address decoding? Since the host already doesn't need to read THR or write RBR they can be safely combined. Some UARTs call this a DATA register instead.
My sweet summer child… this is backwards compatibility to the I/O register set of NatSemi/Intel's 8250 UART chip…
…from 1978.
https://en.m.wikipedia.org/wiki/8250_UART
The definitions are correct, look up an 16550 datasheet if you want to lose some sanity :)
Oh damn, thanks!
In school we were taught that the OS does the printf. I think the professors were just trying to generalize to not go on tangents. But, once I learned that no embedded libc variants had printf just no output path, it got a lot easier to figure out how to get it working. I wish I knew about SWO and the magic of semihosting back then. I don't think those would be hard to explain and interestingly it's one of the few things students asked about that in the field I'm also asked how to do by coworkers (the setting up _write).
> But, once I learned that no embedded libc variants had printf just no output path
Did you mean "once I learned that no, embedded libc variants have printf"?
To clarify as I had to check, embedded libc variants do indeed have some (possibly stripped-down) implementation of printf and as you say they just lack the output path (hence custom output backends like UART, etc).
Sorry yes I had a comma there and then deleted it. I don't remember why but I remember doing it.
I am coding RISC-V assembly (which I run on x86_64 with a mini-interpreter) but I am careful to avoid the usage of the pseudo-instructions and the registers aliases (no compressed instruction ofc). I have a little tool to generate constant loading code, one-liner (semi-colon separated instructions).
And as a pre-processor I use a simple C preprocessor (I don't want to tie the code to the pre-processor of a specific assembler): I did that for x86_64 assembly, and I could assemble with gas, nasm and fasmng(fasm2) transparently.
What's wrong with compressed instructions?
I don't feel comfy using duplicate instructions for a 'R'educed instruction set.
That said, I know in some cases it could increase performance since the code would use less memory (and certainly more things which I don't know because I am not into modern advanced hardware CPU micro-architecture design).
It's just an alternative encoding for exactly the same instructions, it does not make the ISA more complex.
If you are writing assembly you probably are using compressed instructions already since your assembler can do the substitions transparently, e.g.
Example: https://godbolt.org/z/MG3v3jx7P (the disassembly shows addi but the instruction is only two bytes).They offer a nice reduction in code size with basically no downsides :)
I explicitely do disable register ABI alias names, pseudo-instructions and transparent "optimizations", because I run the RISC-V binary in my own x86_64 assembly written little RISC-V machine code interpreter which does support only the core instructions (and a linux syscall translation layer). I may start to add the compressed instructions someday though.
220k just to include studio? That's insane. I have 12k and still do IO. Just without the overblown stdio and sbrk, uart_puts is enough. And only in DEBUG mode.
I thought this was going to talk about how printf is implemented. I worked with a tiny embedded processor that had 8k imem, and printf is about 100k alone. Crazy. Switched to a more basic implementation that was around 2k, and ran much,much faster. It seems printf is pretty bloated, though I guess typical people don't care.
In most C standard libraries intended for embedded applications, including newlib, there is some configuration option to provide a printf that does not support any of the floating-point format specifiers.
That is normally enough to reduce the footprint of printf by more than an order of magnitude, making it compatible with small microcontrollers.
I implemented a secure printf_s and its API is the problem. You cannot dead-code eliminate all the unused methods. And it's type unsafe. There are much better API's to implement a safe printer with all the formatting options still. format is not one of them
The only hack I could think of is having the compiler front end generate calls to different functions based on the content of the format string, similar to how some compilers replace memset with a 32-bit memset based on the type of the destination pointer
And it all falls apart as soon as a format string cannot be known at compile time.
> The only hack I could think of is having the compiler front end generate calls to different functions based on the content of the format string
Compilers do that, at least for the simple case of constant strings; gcc can compile a printf call as puts. See https://stackoverflow.com/questions/60080021/compiler-change...
That's the only one I've ever seen, and it's really just checking for the absence of '%'.
What if it could convert:
printf("parsed: %s to %i\n", x, y);
To:
puts("parsed: "); puts(x); puts(" to "); _puti(y); putc('\n');
Well. Then we'd have a lot more function calls overhead. Maybe something like:
_printf_nofloat("parsed: %s to %i\n", x, y);
It’s not the only one it uses.
It also removes calls to printf("") and changes single character printf calls to putchar calls
See https://github.com/gcc-mirror/gcc/blob/ef32bd8c866a1b8a97f62...
I always felt with these kinds of things you strip out `stdio.h` and your new API/ABI/blackbox becomes `syscall` for `write()`, etc.
If you already are doing bare metal, you probably should think twice if you need a printf, may you just need inb and outb.
I've used avrlibc's printf quite a lot. I always thought it was silly that Arduino never did the simple task of plumbing up to the UART
It’s a feature to rewrite your OS kernel on the fly.
It's 1990, maybe 1999, in embedded land.
honestly love reading about this stuff - always makes me realize how much gets glossed over in school. you think modern cpus and all the abstraction layers help or just make things messier for folks trying to learn the real basics?
I was very confused by the title, expected someone writing their own printf — i.e. the part that parses the format string, grabs varargs, converts numbers, lines up strings, etc.
I'd have called it "Bare metal puts()" or "Bare metal write()" or something along those lines instead.
(FWIW, FreeBSD's printf() is quite easy to pluck out of its surrounding libc infrastructure and adapt/customize.)
Is it? Could you elaborate/provide links to examples of this?
What customization would it support? Say, compared to these options:
https://github.com/eyalroz/printf?tab=readme-ov-file#cmake-o...
> Is it? Could you elaborate/provide links to examples of this?
https://github.com/FRRouting/frr/tree/master/lib/printf
Disclaimer: my work.
Customised to support %pHX, %pI4, %pFX, etc. - docs at https://docs.frrouting.org/projects/dev-guide/en/latest/logg... for what these do.
> What customization would it support?
I don't understand your question. It's reasonably readable and understandable source code. You edit the source code. That's the customisation?
> Say, compared to these options: https://github.com/eyalroz/printf?tab=readme-ov-file#cmake-o...
First, it is customary etiquette to indicate when linking your own code/work.
Second, that is not a POSIX compatible printf, it lacks support for '%n$' (which is used primarily for localisation). Arguably can make sense to omit for tiny embedded platforms - but then why is there FP support?
Third, cmake and build options really seem to be overkill for something like this. Copy the code into the target project, edit it. If you use your own printf, you probably need a bunch of other custom stuff anyway.
Fourth, the output callback is a reasonable idea, but somewhat self-contradictory. You're bringing in your own printf. Just adapt it to your own I/O backend, like libc has FILE*.
> You edit the source code. That's the customisation?
I meant, customization where you don't have to write the customized code yourself, just choose some build options, or at most set preprocessor variables.
> First, it is customary etiquette to indicate when linking your own code/work.
You're right, although I was only linking to the table of CMake options. And it's only partially my code, since I'm the maintainer rather than the original author
> You're bringing in your own printf. Just adapt it to your own I/O backend, like libc has FILE.
One can always do that, but - with the output callback - you can bring in an already-compiled object, which is sometimes convenient.
> If you use your own printf, you probably need a bunch of other custom stuff anyway.
My personal use case (and the reason I adopted the library) was printf deficiencies in CUDA GPU kernels. And - I really needed nothing other than printf functions. Other people just use sprintf to format output of their mostly, or wholly, self-contained functions which write output to buffers and such. Different strokes for different folks etc.
But - I will definitely check out the link.
> Second, that is not a POSIX compatible printf, it lacks support for '%n$' (which is used primarily for localisation).*
That is true. But C99 printf and C++ printf do not support that either. ATM, the aim is completing C99 printf support (when I actually work on the library, which is not that often). So, my priority would be FP denormals and binary FP (with "%a"), before other things.
> * Arguably can make sense to omit for tiny embedded platforms - but then why is there FP support?*
It's there because people wanted it / needed it; and so far, there's not been any demand for numbered position specification.
> I meant, customization where you don't have to write the customized code yourself, just choose some build options, or at most set preprocessor variables.
Honestly, if you're shying away from customising an 1-2kloc piece of code, you probably shouldn't be using a custom printf().
Case in point: function pointers are either costly or even plain unsupported on GPU architectures. I would speculate that you aren't using the callbacks there?
> Honestly, if you're shying away from customising an 1-2kloc piece of code, you probably shouldn't be using a custom printf().
Well, it was good enough for the arduino SDK to adopt: https://github.com/embeddedartistry/arduino-printf
> * function pointers are either costly or even plain unsupported on GPU architectures.*
When you printf() from a GPU kernel, your performance is shot anyway, so performance is not a consideration. And - function pointers work, as long as they all get resolved before runtime, and you don't try to cross CPU <-> GPU boundaries.
> > Honestly, if you're shying away from customising an 1-2kloc piece of code, you probably shouldn't be using a custom printf().
> Well, it was good enough for the arduino SDK to adopt: https://github.com/embeddedartistry/arduino-printf
Well, they didn't shy away from customizing it quite a bit ;)
To be clear I was trying to say it doesn't make too much sense to try to package this as an independent "easy to use" "library" with a handful of build options. Not that it's somehow not "good enough".
Put another way: a situation where you need/want a custom printf is probably a situation where a package like this doesn't exactly help you anyway and you'll need to muck with it regardless. But the code can be used. Which is exactly what the repo you linked did.
FreeBSD’s printf is my goto, too! It’s indeed enormously simple to pluck out, instantly gives you full-featured printf, and has added features such as dumping memory as hex.
Funnily enough we're not even referring to the same one, the hexdump thing is in FreeBSD's kernel printf, I was looking at the userspace one :). Haven't looked at the kernel one myself but nice to hear it's also well-engineered.
(The problem with '%D' hexdumps is that it breaks compiler format checking… and also 'D' is a length modifier for _Decimal64 starting in ISO C23… that's why our hexdump is hooked in as '%.*pHX' instead [which still gives a warning because %p is not supposed to have a precision, but at least it's not entirely broken.])
[flagged]
What's the state of the art for cross-compiling in $CURRENTYEAR?
I just run gentoo and follow their cross-compile guide, as well as set up distcc; but that's for system packages.
TBH I'd use qemu if I had to make something work for arbitrary code.
https://wiki.gentoo.org/wiki/Crossdev
But there are others.
Well, I know it's pretty easy in Debian. (It's not completely pain-free if you need unpackaged third-party libraries and/or if you are cross-compiling from one uncommon architecture to another.)
Honesty probably zig cc.
> What's the state of the art for cross-compiling in $CURRENTYEAR?
Poopy garbage dog poop.
glibc is a dumpster fire of bad design. If you want to cross-compile for an arbitrarily old version of glibc then... good luck. It can be done. But it's nightmare fuel.
Is the cross-compilation story any better with musl?
tbh I haven't gone too deep because glibc is such a standard :(
but I can answer with reasonable confidence "musl surely has other problems, but not this one". It's a nice, clean, simple, single set of headers and source files. Very nice.
What sort of roadblocks do you run into? Just to get an idea for the flavor of problems.
Well there's two flavors to this. Building glibc and building a program that links against glibc. They're not entirely the same. And you'd think the latter is easier. But you'd be wrong!
It should be trivial to compile glibc with an arbitrary build system for any target Linux platform from any OS. For example if I'm on Windows and I want to build a program that targets glibc 2.23 for Ubuntu on x86_64 target that should be easy peasy. It is not.
glibc should have ONE set of .c and .h files for the entire universe. There should be a small number of #define macros that users need to specify to build whatever weird ass flavor they need. These macros should be plainly defined in a single header file that anyone can look at.
But glibc is a pile of garbage and has generated files for every damn permutation in the universe. This is not necessary. It's a function of bad design. Code should NEVER EVER EVER have a ./configure step that generates files for the local platform. EVER.
Read this blog post to understand the mountains that Zig moved to enable cross-compilation. It's insane. https://andrewkelley.me/post/zig-cc-powerful-drop-in-replace...
>glibc should have ONE set of .c and .h files for the entire universe. There should be a small number of #define macros that users need to specify to build whatever weird ass flavor they need. These macros should be plainly defined in a single header file that anyone can look at.
Code generation aside, this is not really a great way to do it either. The build should include target-specific files, as opposed to creating a maze of ifdefs inside the code.
> The build should include target-specific files, as opposed to creating a maze of ifdefs inside the code.
Hard disagree. What makes you think it's a maze of ifdefs?
Compiling a library should be as simple as "compile every .c/.cpp file and link them into a static/shared lib". The nightmare maze is when you don't do that and you need to painfully figure out which files you should and shouldn't include in this particular build. It's horrible.
Far far far simpler is to stick with the rule "always compile all files". It's very simple to ifdef out an entire file with a single, easily understood ifdef at the top.
I do agree you don't want the middle of a file to be a fully of 10 different ifdef cases. There's an art to whether to switch within a file or produce different files. No hard and fast rule there.
Fundamentally you put either your branching logic in the code or in the build system. And given that source files should be compatible with an unbounded array of potential build systems it is therefore superior to solve the problem once in the code itself.
I am currently trying to get the Zig glibc source/headers to compile directly via clang and trying to figure out which files to include or not include is infuriatingly unclear and difficult. So no, I strongly and passionately disagree that build-system is where this logic should occur. It's fucking awful.
gcc and glibc SDKs are hell beyond anything sane.
Like it is done on purpose.
You're getting downvoted but you're not wrong. Well, it's not on purpose per se. It's just really really bad design from the 80s when we didn't know better. And unfortunately its design hasn't been fixed. So we have to deal with almost 40 years of accumulated garbage.
Aren't there some ideologically driven decisions to make proprietary extensions harder/impossible?
They would be illegal everywhere the GNU GPL can be enforced.
Indeed, that why HN comments are kind of dead (AIs, zealots with one billion accounts behind VPNs, etc). Sometimes I do still try to give an honest, but unpleasant, opinion/fact though. I should stick to raw neutral communication and information publishing.