Complete C standard library
Posté le 19/08/2018 19:31
Motivation
Until now there was no complete C standard library (aka
libc) available for the Casio calculators. Although some parts of this library have been provided by fxlib and
gint, there was no libc implementation complying with the standard and compatible with the sh3eb architecture ready to use.
To change that, I decided to port
newlib to the Casio CPU. Newlib is an alternative libc implementation intended for use on embedded systems.
Alpha
Follow this link and click the download button in the top right corner:
>>> v1.1 <<<
Instructions on how to install newlib alongside with gcc (big shout-out to Lephé):
Compiler sous Linux avec GCC
Features for Casio fx9860g calculators:
* C standard library
libc
→
printf implementation to print text to the display
→ Dynamic allocation of memory using
malloc and
free
→ Memory manipulation using
memcpy,
memcmp,
memset etc.
→ String manipulation using
strcpy,
strcmp,
strstr,
strtok
→ ...
* Math library
libm
→ Floating point arithmetics
→ ...
* Automatic library and include path recognition after installation
* Basic Casio features:
→ implementation of
GetKey,
Bdisp_AllClr_DDVRAM,
Bdisp_PutDisp_DD,
Print and
locate without fxlib (but you can use it if you want)
Code
To contribute or get all those bleeding edge features, see the code including all further information:
libc (my GitLab repository)
The project you find in my repository is a fork of
the official newlib repository. To make it easier for everyone to follow, I try to keep a clean
git history. That means that all my changes are located on a dedicated branch with meaningful commits.
I also try to keep the changes to the upstream library minimal. That increases maintainability a lot.
Contributing
If you have a ideas, feature request, found a bug or simply want to contribute, just send me a message and you're in! You can also create
Issues and
Merge Requests directly in the repository. As in every OpenSource project: merge requests welcome!
Citer : Posté le 20/08/2018 20:32 | #
sh3eb-elf-gcc -o build/cryptfx.elf build/cryptfx.cpp.o -m3 -mb -ffreestanding -Iinclude -O2 -std=c11 -T ld/cryptfx.ld -Llib -lc-lgcc -lfx
.../ld: section .eh_frame LMA [0000000000301dd4,0000000000301e27] overlaps section .data LMA [0000000000301dd4,000000000030260f]
.../ld: warning: section`.bss' type changed to PROGBITS
Citer : Posté le 20/08/2018 20:34 | #
I just had the same error this morning
See this message: https://www.planet-casio.com/Fr/forums/topic12970-12--Tutoriel--Compiler-sous-Linux-avec-un-cross-compilateur-gcc.html#156796
Ecrivez vos programmes basic sur PC avec BIDE
Citer : Posté le 20/08/2018 20:56 | #
Unrelated question: is there a simulator for the fx9860GII like the one shipped with the Casio SDK available for linux? If not, do you know any other way to debug my programs?
Since I moved to Linux I only trust testing on my calculator. If you have one, note that the workflow is considerably faster than using external software (even more when using a VM) because you can upload the program from the Makefile using Cakeisalie5's p7 utility.
Do you reckon it would be useful to create a repo for a test program? It would make tests easier to reproduce, and we could check the same programs under fxlib/gcc, gint or any other runtime to make sure we don't screw up in one of them. I'm not really sure. What you do think?
sh3eb-elf-gcc -o build/cryptfx.elf build/cryptfx.cpp.o -m3 -mb -ffreestanding -Iinclude -O2 -std=c11 -T ld/cryptfx.ld -Llib -lc-lgcc -lfx
.../ld: section .eh_frame LMA [0000000000301dd4,0000000000301e27] overlaps section .data LMA [0000000000301dd4,000000000030260f]
.../ld: warning: cannot find entry symbol initialize; defaulting to 0000000000300200
.../ld: warning: section`.bss' type changed to PROGBITS
Ok, so this is pretty normal. There are three things here.
1. Section .eh_frame and section .data overlap. This is due to section .eh_frame not being mentioned in the linker script, so ld puts it somewhere seemingly random. We can either explicitly load this section into a ROM or RAM area or discard it. Since it contains asynchronous stack unwind tables that are only useful for the C++ exception mechanism, I suggest we discard it. To do it, you can add a /DISCARD/ section at the end of the linker script and put *(.eh_frame) in it, following what's in gint's linker scripts.
2. Entry symbol initialize cannot be found. You seem to have forgotten to include crt0.s in your build and tried to link right away. Make sure you include it when you link.
3. There is a certain kind of symbol in the .bss section which causes it to turn to PROGBITS. This is technically not a problem. Variable _bssdatasize from crt0.s is known to cause this problem because it's in the .bss section. Usually we just ignore it, as it won't cause any runtime bug.
That's good to hear!
I'm currently in the process of building binutils 2.31.1 with gcc 8.2.0 for target sh3eb-elf, with the required options to target newlib. I should be able to conduct some tests when it's done.
Citer : Posté le 21/08/2018 00:38 | #
Thanks a lot! I'll continue to work on this tomorrow!
Yes, I want to create a remote repository for testing, but I'm not ready yet >_<. This is quite some work still xD.
I previously mapped sh3eb to sh. Now I managed to create a dedicated arch for sh3eb in order to be able to change code cleanly. Using the autotools was quite a hassle...
I think the next step is to look at the linker scripts. I'll probably try to stay close to sh3lcevb.ld, but we'll see. Tbh I do not really see how the newlib takes it's own linker script in account. Apparently, its own crt0.S is linked by default. This is the part I fear the most
@Lephé Does gint use a crt0.s, too? Where is it, I couldn't find it (yet I could create addins without any problem)?
Another challenge is to configure newlib's headers. If you look here, you'll see that one has to set the endianess of floating point variables etc. (Even if we have no FPU, floats are emulated, n'est-ce pas?)
The section for sh in ieeefp.h is:
#ifdef __LITTLE_ENDIAN__
#define __IEEE_LITTLE_ENDIAN
#else
#define __IEEE_BIG_ENDIAN
#endif
#if defined(__SH2E__) || defined(__SH3E__) || defined(__SH4_SINGLE_ONLY__) || defined(__SH2A_SINGLE_ONLY__)
#define _DOUBLE_IS_32BITS
#endif
#endif
Citer : Posté le 21/08/2018 07:44 | #
You've already built it and made some tests! This is impressive!
Now I'm lost. Why would you want to write a linker script? I thought the user of the libc would use the one provided with their runtime system, be it fxlib/gcc or gint or whatever. Besides, if you try to link an add-in with something close to sh3lcevb.ld, I reckon a lot of data will be missing.
Yes it does, but it's not written in assembler anymore. The current initialization takes place in src/core/start.c.
You're absolutely right, floating points are handled by libgcc. Looking at your linked documentation, I'm pretty sure we only need __IEEE_BIG_ENDIAN since the libgcc implementation is certainly IEEE 754-compliant. As for the SH2/SH3E/SH4/SH2A switch, it corresponds to SuperH variants that have an FPU, so we don't care.
Ajouté le 21/08/2018 à 17:09 :
It was a good occasion so I reworked my gcc building tutorial and I added an only-if-you-now-what-you're-doing section about building newlib.
Of course I had to test it so my sh3eb-elf compiler is now up-to-date and ready to try out the newlib port.
Citer : Posté le 21/08/2018 18:28 | #
It was a good occasion so I reworked my gcc building tutorial and I added an only-if-you-now-what-you're-doing section about building newlib
Wow, already? That's an honor!
Now I'm lost. Why would you want to write a linker script?
You are right. First I thought the default linker script is shipped with the C standard library but it seems it's provided by binutils (sh3eb-elf-ld --verbose). Sorry for the confusion.
As for testing: I created a repo for the first tests using fxlib. I obviously still have problems xD. Simply call make for building. You can also call make upload which does not build but upload the .g1a file to your fx9860g (requires g1a-wrapper and CasioUsbUploader to be in your $PATH). Please don't judge me for my rudimentary Makefile xD.
Later I'll upload a new version of my newlib. To keep my git history clean I'll rewrite history, so you might need a little more power to pull:
git fetch
git reset --hard origin/master
Citer : Posté le 21/08/2018 18:57 | #
I might not update this tutorial in several years (2 years since last update), so better now than never.
I'm still confused, in fact!
I'm not used to this kind of Makefile, but it should still work. I can build it. I sent a small commit with...
- A different icon (as a developer of gint I have it on my calculator so I couldn't differentiate xD)
- A slightly different definition of CASIOUPLOADER to allow using Cakeisalie5's recent p7 utility
- A .gitignore to avoid indexing the build directory by mistake
There's something strange going on as at the second loop iteration the text turns into "rld!o World!" on my machine, I haven't looked into the details but I guess there are pointers moving.
Citer : Posté le 21/08/2018 19:09 | #
I'm surprised that you have to permissions to commit. Nonetheless, thanks!
I won't provide a linker script (unlike I initially thought) because libc typically doesn't provide any linker script (if I understood that correctly). So the user will use the fxlib/gint one.
The rld!o World! happens for me, too. That's what I meant with "having problems" xD
Ajouté le 21/08/2018 à 19:14 :
Found the error, fixed it.
Ajouté le 21/08/2018 à 19:20 :
Unfortunately, malloc still returns 0.
Ajouté le 21/08/2018 à 19:27 :
@Lephé are you sure that when you describe how gcc is built the second time in your tutorial that --without-headers should be passed?
Ajouté le 21/08/2018 à 20:30 :
Hmm, I'm not really sure how to avoid conflicts between newlib and fxlib (malloc etc.). I'll try gint (compat) for now because I can modify it in case of conflicts.
@Lephé Is the printing of text already implemented? I didn't find a dtext function or anything similar
Citer : Posté le 21/08/2018 20:31 | #
Oh, sorry, I did not check the permissions on the GitLab to see if you intended this. I don't have a very deep Git experience and I tend to commit on Planète Casio projects rather than using pull requests. As for the rights, it happens that I am one of the GitLab administrators so I have virtually every right.
What do you think about putting the fxlib/gcc linker script in your ld directory then?
Nice! I had a look at the Git history but apparently you already rewrote it. I was able to fetch the new content thanks to your commands.
This is clearly not the simplest function. I have a deeper question about it: what do we do about the system's malloc() functions and heap?
I admit that I had overlooked that when writing the tutorial. Turns out --without-headers is a libgcc option and I genuinely want my libgcc to be independant from the headers because I might build applications without a libc, such as the gint control add-in. So after checking it, yes I'm positive. Thanks for pointing that out
Ajouté le 21/08/2018 à 20:32 :
@Lephé Is the printing of text already implemented? I didn't find a dtext function or anything similar
I was about to start doing graphical functions today, so not now. But you can freely use the fxlib drawing functions.
Citer : Posté le 21/08/2018 20:43 | #
I did not check the permissions on the GitLab to see if you intended this
It's ok, since you know what you are doing, feel free to commit (I can still rewrite history if I want). I don't know how firm you are with the git/gitlab/github workflow, so just a little explanation. If you want to contribute to a open source project, in the "real world" you create a dedicated feature branch, commit there and create a "merge request" (gitlab) or "pull request" (github). After the code review, the maintainer (that would be me) merges the branch into master (or gives feedback so you can change some pieces).
But again, feel free to commit
For my test projects, I have the fxlib/gint linker script the ld directory of my local repo. Without linker script, I get linker errors. I'm confused, did I miss anything?
I did not rewrite history, here is the fix. I don't really care about the history of my test repos, only the newlib repo should stay clean xD.
To be honest, I have no idea yet about what to do with anything. I'm also not sure if the malloc which doesn't work is the symbol provided by fxlib or by libc. I did not manage to rename the fxlib symbols...
Ajouté le 21/08/2018 à 20:45 :
Don't feel pressured xD
Citer : Posté le 21/08/2018 20:55 | #
Thanks for the reminder! Is it okay to create a new branch directly on your repo without cloning?
Oh wait, I was the one to miss out something. Your linker script is quite minimal. As I have only looked at gint's complex linker scripts for a while, I was under the impression that the fxlib/gcc script was longer and more complex than that. So I was suggesting that you used it, but you were already! Sorry.
So I definitely need to expand it to add mentions to fxlib sections, it would be much cleaner.
(I looked at the main repo's history actually... nvm -_-")
You can explicitly remove _malloc.robj from fxlib.a if you care. But I don't see how the system could refuse to allocate such a small piece of heap.
If you want to be sure, you can check the address of the malloc() function in your linked executable with objdump -t and disassemble the code in the area with objdump -d. Almost all fxlib functions are syscalls, meaning that they load a magic value into r0 and jump (not a subroutine call) at a fixed address. You will instantly be able to tell whether you're running newlib code.
Citer : Posté le 21/08/2018 21:19 | #
Is it okay to create a new branch directly on your repo without cloning?
What I meant is: You clone my repository (which we call upstream). Now you have a local copy. Then you upload it to your own remote repository (which is called origin). Then you create a branch locally and commit your changes there. After pushing them to origin, you can create a merge request. Not particularly intuitive, I know xD.
All this hassle ensures, that the maintainer of the upstream repository can review the code before it's really checked in. Since you are a co-developer, feel free to commit directly. I would have given you the permissions, but you know... xD.
You can explicitly remove _malloc.robj from fxlib.a if you care.
Yeah, I think I'll try that tomorrow.
Unrelated: so I read something about UART if I recall correctly. Did anyone consider putting an esp8266 into a Casio?
Ajouté le 21/08/2018 à 21:31 :
Oh and if you have a copy of upstream, the repository is called fork. (origin and upstream are more the names for the locations (urls), but sometimes you also call the repositories so)
Citer : Posté le 21/08/2018 21:40 | #
That fits what I (though I) knew better! So basically :
1. I clone the project and I have both a local copy and my own forked repository
2. I push changes to a new branch of my own repo
3. I submit a merge request.
Is that right?
Not sure what the specifics of "UART" are (never used this term before) but we do have a serial connection and it can theoretically operate in an asynchronous mode. @Zezombye dit put a Bluetooth module in it, communicated with his phone and browsed Reddit from his calculator. xD
Citer : Posté le 21/08/2018 21:50 | #
Yes, I did. But I broke the ESP testing it on an Arduino, configured in 5V and not 3.3V xD
The ESP8266 is a low-energy WiFi adaptator who use UART (Universal Asynchronous Receiver Transmitter, aka Serial Port) to dialog with other devices. The goal is to have a calc connected to the Internet
Since Lephe also built a simple html interpreter known as WebCalc, it should be possible to load some webpages oncalc.
If we consider a server working as a proxy, that can simplify any webpage to something compatible with WebCalc, it's possible to access all the Web directly oncalc.
and it can theoretically operate in an asynchronous mode.
Serial is by-design an asynchronous protocol. To have synchronous communication, you need to share a clock between parties (like I2C's SCL (Shared CLock) wire)
Citer : Posté le 21/08/2018 21:54 | #
Oh god, I'm so dumb it's laughable. We did play with an esp8266 in our System/Network course last semester, although I did not do the manipulations myself. >_<
Yes, that would be worth the trip. The drawback, as always, is that pretty much no one will ever use it. x)
Edit : Well noted for the asynchronous thing. I wasn't sure because the "default" serial communication peripheral module is called the SCIF and SH7724 has others called Asynchronous SCIFs. Kinda bugged me.
Citer : Posté le 21/08/2018 21:58 | #
Yes that's right!
UART is a serial interface (just like SPI, I2C are digital interfaces). If you say "serial connection", you only mean that the bits of one byte are not sent at the same time over 8 wires (parallel) but over one wire one after another (serial). UART specifies one way of serial communication (frequency, error handling, parity bits, synchronization, voltage levels etc.)
Thats the pragmatic way of explaining. UART is basically the most common way of serial interface. There are probably better explanations on the internet
Citer : Posté le 21/08/2018 21:59 | #
@Lephe : You're right, but SCI(F?) stands for "Serial Communication Interface", not "Synchronous Communication Interface"
Citer : Posté le 21/08/2018 22:01 | #
Aha tweaking calculators is really quite fun xD
You're right, but SCI(F?) stands for "Serial Communication Interface", not "Synchronous Communication Interface"
Well, the existence of "Asynchronous SCIFs" suggests that the others are synchronous... that's about as far as my logic thinking goes
Citer : Posté le 21/08/2018 22:05 | #
While the serial port of your pc is effectively uart, i wouldn't call it just "serial", because there are many many more serial protocols out there.
As for the (a)sync part... Don't forget abou USART xD
Citer : Posté le 21/08/2018 22:08 | #
When I say "SCIF" I actually refer to serial communication peripheral modules of the calculator's MPU, so it should not be anything too fancy or too complicated.
Citer : Posté le 22/08/2018 20:11 | #
Today I managed to setup some syscalls listed here. That means for now I'm testing newlib without any library to avoid conflicts. Later I'll add the support for fxlibc/gint.
Do you think it makes sense to replace the libc implementations of free, malloc, memcmp, memcpy, memset, strcat, strcmp, strlen, strncat, strncmp, strncpy, strrchr etc. with syscalls?
Ajouté le 22/08/2018 à 20:15 :
This implies that I call the addresses directly, gint will probably not have any control over it...