Complete C standard library
Posté le 19/08/2018 19:31
Motivation
Until now there was no complete C standard library (aka
libc) available for the Casio calculators. Although some parts of this library have been provided by fxlib and
gint, there was no libc implementation complying with the standard and compatible with the sh3eb architecture ready to use.
To change that, I decided to port
newlib to the Casio CPU. Newlib is an alternative libc implementation intended for use on embedded systems.
Alpha
Follow this link and click the download button in the top right corner:
>>> v1.1 <<<
Instructions on how to install newlib alongside with gcc (big shout-out to Lephé):
Compiler sous Linux avec GCC
Features for Casio fx9860g calculators:
* C standard library
libc
→
printf implementation to print text to the display
→ Dynamic allocation of memory using
malloc and
free
→ Memory manipulation using
memcpy,
memcmp,
memset etc.
→ String manipulation using
strcpy,
strcmp,
strstr,
strtok
→ ...
* Math library
libm
→ Floating point arithmetics
→ ...
* Automatic library and include path recognition after installation
* Basic Casio features:
→ implementation of
GetKey,
Bdisp_AllClr_DDVRAM,
Bdisp_PutDisp_DD,
Print and
locate without fxlib (but you can use it if you want)
Code
To contribute or get all those bleeding edge features, see the code including all further information:
libc (my GitLab repository)
The project you find in my repository is a fork of
the official newlib repository. To make it easier for everyone to follow, I try to keep a clean
git history. That means that all my changes are located on a dedicated branch with meaningful commits.
I also try to keep the changes to the upstream library minimal. That increases maintainability a lot.
Contributing
If you have a ideas, feature request, found a bug or simply want to contribute, just send me a message and you're in! You can also create
Issues and
Merge Requests directly in the repository. As in every OpenSource project: merge requests welcome!
Citer : Posté le 31/08/2018 23:16 | #
Couldn't you just put the function declaration inside the define?
Ecrivez vos programmes basic sur PC avec BIDE
Citer : Posté le 31/08/2018 23:16 | #
As a quick idea (I did not look up the files right now, it's getting late), you can swiftly #undef calloc before making your definition in the source file. But considering how you overrode functions just by providing machine-specific versions earlier, why don't you define the syscall under the name calloc() in the sh3eb-specific directory?
Ajouté le 31/08/2018 à 23:16 :
Zezombye : this would not work since calloc(size_t num, size_t size) would also trigger the macro.
Citer : Posté le 31/08/2018 23:18 | #
I don't see why this wouldn't work, a call to calloc would be redirected to calloc_ptr. (and obviously, remove the declaration in the header)
Ecrivez vos programmes basic sur PC avec BIDE
Citer : Posté le 31/08/2018 23:26 | #
@Zezombye
Good idea, but as Lephé pointed out not only the calls of calloc() would be replaced, but also its declaration inside stdlib.h. Since at this point, calloc_ptr() does already exist, I'd get a redefinition error.
@Lephé
Frankly I have no idea why it works for malloc and not for calloc xD. I fear #undef calloc will not work either since calloc() is not a macro, but a function. Unfortunately in C functions cannot be undefined (or undeclared).
Ajouté le 31/08/2018 à 23:30 :
I did not look up the files right now, it's getting late
I couln't agree more, see you tomorrow xD
Citer : Posté le 31/08/2018 23:33 | #
If you remove the declaration in the header, you shouldn't get a redefinition error, because it isn't defined
Ecrivez vos programmes basic sur PC avec BIDE
Citer : Posté le 31/08/2018 23:44 | #
Oh right! When I tried it out first, it didn't work because _calloc_ptr() is actually a function pointer. Its definition has consequently a different type than the calloc declaration. Creating a real function which replaces calloc and calls _calloc_ptr should do the trick! I'll try that out tomorrow, thanks xD.
Citer : Posté le 01/09/2018 08:53 | #
I'm confused. You just said earlier that you wanted to define calloc() through
That, of all things, makes calloc a macro. If you #undef calloc at some point in a source file, the translation will stop and mentions of calloc will remain as is. Especially its definition.
Let's clear things up:
- For the declaration in stdlib.h, you need only put the define after the declaration.
- For the definition in mallocr.c, you can hide the macro with #undef during the definition.
Ajouté le 01/09/2018 à 08:54 :
Or maybe you mean that this :
x(1)
may not translate to y(1)? It does, though.
Citer : Posté le 01/09/2018 15:09 | #
I'm confused. You just said earlier that you wanted to define calloc() through
Yes, I want to #define calloc() in machine/stdlib.h which is target-specific. However, I do not want to use #undef in stdlib.h which is not target-specific.
Also, mallocr.c is not target-specific but since it does not include stdlib.h, it is not affected by the #define.
The solution Zezombye suggested worked. In machine/stdlib.h, I simply did not declare _calloc_ptr() so I do not get a redeclaration error. Instead I had to declare calloc (because the declaration in stdlib.h is will be replaced).
That way I do not need to touch stdlib.h which makes me very happy xD. So far I did not change any original code. (I only copied some code for my target which is fine).
Citer : Posté le 01/09/2018 15:17 | # |
Fichier joint
Plus, as you may know, one must be able to use the standard library functions without including the related headers. Here's the example given by the ISO C99 standard:
Mon blog ⋅ Mes autres projets
Citer : Posté le 01/09/2018 15:22 | #
I think I need some more explanations. Please bear with me...
If you remove the declaration in the header, you shouldn't get a redefinition error, because it isn't defined
This doesn't make any sense...? If you have a redefinition error, it's because you've defined a symbol twice. Removing a declaration will not change a thing, and I can't be mistaken here, because there are no definitions in a header.
Yes, I want to #define calloc() in machine/stdlib.h which is target-specific. However, I do not want to use #undef in stdlib.h which is not target-specific.
Ok, so I was about to suggest that you add calloc() to machine/stdlib.h, letting the macro rewrite the declarations in stdlib.h. Which is what you did, right?
This "re-declaration" issue bugs me though. How can there even be a re-declaration error?
Citer : Posté le 01/09/2018 15:32 | #
@Lephé
You are definitely right, I confused calloc with calloc_r which contains the actual logic. I also wanted to keep the explanation simple :D. To wrap it up: there is no "redeclaration" error and I had an actual redefinition error which I could resolve (Zezombye's idea was great, his suggestion gave me the hint I needed).
In the end it does not really matter, because Cakeisalie5 is totally right.
@Cakeisalie5
I did not know this at all. I guess #defining is not the thing I actually should do here. The standard is very clear about that.
I'll look into that some more.
Citer : Posté le 01/09/2018 15:34 | #
What about defining a weak calloc() that calls calloc_ptr() ?
Citer : Posté le 01/09/2018 15:50 | #
It will be overwritten by the original calloc
Citer : Posté le 01/09/2018 16:09 | #
So a normal function in the machine-dependent files overwrites the default, but a weak function in the machine-dependant files does not? Hmm...
Can you remind me of the reason you did not name the syscall calloc() in the first place?
Citer : Posté le 01/09/2018 16:23 | #
Firstly, a weak function never overwrites a normal function.
As I said, I have no idea why the definition of malloc here taked precedence over the original definition here. To be honest, I would actually expect a redefinition error but there is none.
If I define a function named calloc on the other side I do get a redefinition error!? There is something I am missing...
@Cakeisalie5
Do you have any idea?
Citer : Posté le 01/09/2018 16:50 | #
I would expect this, but I figured there could be some sort of file-level override from the machine-dependent directory, so that the machine-independant code is actually not compiled... nvm.
I pulled the repository and found merge conflicts because there are autoconf-generated files in the index... is this intentional?
I settled on git fetch followed by a hard reset to the origin/sh3port branch... not sure what happened there.
I analyzed the build log on my system and look what I found:
There we have a file-level replacement of newlib/libc/stdlib/malloc.c by the version available in the building directory, which is sh3eb-elf/newlib/libc/stdlib at this point.
This does not explain everything of course, but my theory may have some truth in it.
Citer : Posté le 01/09/2018 22:30 | #
I pulled the repository and found merge conflicts because there are autoconf-generated files in the index... is this intentional?
I only modified some autotool generated files, but I did not add any new ones. The merge conflicts are due to rewritten history xD. Now you see why some consider it bad practice. Please excuse the inconvenience, but for this project, a clean history is more important to me than traceability.
There we have a file-level replacement of newlib/libc/stdlib/malloc.c by the version available in the building directory, which is sh3eb-elf/newlib/libc/stdlib at this point
Thank you, that's indeed a good hint. After a quick grep, I found the following:
./newlib/libc/stdlib/Makefile.in:826
I'll look into this some more another day.
Citer : Posté le 01/09/2018 23:16 | #
Not a problem, really. It's rather the opposite since it forces me to actually understand how Git works under the hood.
By the way, the same "file overrides" happen with files in the sys directory. So this line is where your malloc() system call from newlib/libc/sys/sh3eb is being compiled:
Interestingly enough, the malloc.c file that is compiled is from newlib/libc/stdlib (this is the one that you linked earlier), but it's used in a Makefile execution that takes place in sh3eb-elf/newlib/libc/stdlib; it is accessed through the "default-file" mechanism with the test -f:
...
sh3eb-elf-gcc ... `test -f 'malloc.c' || echo '../../../.././newlib/libc/stdlib/'`malloc.c
This is why the source is in newlib/libc/stdlib but the object file is in sh3eb-elf/newlib/libc/stdlib. The same applies for casio_syscalls.c.
I tracked down the object files and confirmed that they both export a global symbol _malloc. They are then linked into static libraries:
- malloc.c goes to sh3eb/newlib/libc/stdlib/lib.a;
- casio_syscalls.c goes to sh3eb/newlib/libc/sys/sh3eb/lib.a.
Then sh3eb/newlib/libc/sys/sh3eb/lib.a is hard-linked into sh3eb/newlib/libc/sys/lib.a, which is confirmed by them sharing the same inode number.
Later on in the build process (around 60% of my log), the libc archive is formed by compiling all these invidual libraries. This takes places in folder sh3eb-elf/newlib/libc:
cd tmp; \
for i in argz/lib.a stdlib/lib.a ctype/lib.a search/lib.a stdio/lib.a string/lib.a signal/lib.a time/lib.a locale/lib.a reent/lib.a errno/lib.a misc/lib.a ssp/lib.a syscalls/lib.a machine/lib.a sys/lib.a; do \
sh3eb-elf-ar x ../$i; \
done; \
sh3eb-elf-ar rc ../libc.a *.o
sh3eb-elf-ranlib libc.a
By re-performing this step (linking the archive into /tmp) I can confirm that both objects are in the archive:
lib_a-casio_syscalls.o
lib_a-malloc.o
It is worth mentioning that stdlib/lib.a and sys/lib.a are only mentioned at that time, not re-used later on (or at least the strings stdlib/liba and sys/lib.a are not used, you see what I mean
The only other occurrences of the string libc.a happen after libm is built and basically it creates libg.a by merging object files from libc.a and libm.a into a single archive; nothing of interest, I believe.
-
So I have Yet Another Theory. It didn't come to my mind before, but you need not get a redefinition error for using the same symbol twice as long as you don't link it twice.
I now basically know that libc.a has two _malloc symbols. During linking, when this symbol goes unresolved, the linker looks into all libraries in order until it finds it. It probably traverses the list of object files in the archive until it finds the symbol, then does some graph traversal to include all the object files it depends on.
So what if the linker just took the first occurrence it found in the library? If you have a look, sh3eb-elf-ar suggests that casio_syscalls.o is before malloc.o for a certain order used to display the object file names:
lib_a-casio_syscalls.o
lib_a-malloc.o
Now this is in fact the good old lexicographic order... which means that casio_syscalls.o is after calloc.o. Rings a bell?
lib_a-calloc.o
lib_a-casio_syscalls.o
Now you might argue that lexicographic order is probably just for display, but a close look at an hexadecimal dump of the archive (and a quick search of the lib_a- string) will show you that these names are in fact stored in lexicographic order in the archive file. This is not unexpected because the build system runs ranlib on all generated archives.
I admittedly got a bit carried on, but hopefully I now have a reasonable idea of why malloc() is overridden but not calloc()!
Citer : Posté le 02/09/2018 13:13 | #
It's rather the opposite since it forces me to actually understand how Git works under the hood.
Usually what I do to rewrite history is an interactive rebase. You can step through the commits and use --amend to change the changes. You can also switch the order of the commits, omit commits entirely etc.
However, when you change a commit, it's SHA hash changes as well.
For example
You clone my remote repository with the commits:
AAAAAAA ---- BBBBBBB ---- CCCCCCC
|
Head, sh3port, origin/sh3port
Now I decide to change BBBBBBB. This is how the remote repo looks like now:
AAAAAAA ---- XXXXXXX ---- CCCCCCC
|
sh3port
When you decide to pull (= fetch+merge), git internally performs a fetch first:
Head, sh3port
|
AAAAAAA ---- BBBBBBB ---- CCCCCCC
\
+- XXXXXXX ---- CCCCCCC
|
origin/sh3port
Afterwards, git internally performs a merge of sh3port into origin/sh3port
This will fail, of course, because XXXXXXX and BBBBBBB most probably conflict which each other.
git reset --hard <commit> forces a branch to point to a specific commit without merging.
Ajouté le 02/09/2018 à 13:16 :
Wow, thank you Lephé, you really dug yourself into this! Unfortunately, a quick test of renaming casio_syscalls.c to _casio_syscalls.c did not change the behavior, yet. However, I have some very nice hints where to look for now. I bet in the end, there is a simple solution for my problem
Citer : Posté le 02/09/2018 14:12 | #
Nice, thanks for the explanation!
Unfortunately, a quick test of renaming casio_syscalls.c to _casio_syscalls.c did not change the behavior, yet.
I'm not sure about the validity of a test with a non-alphanumerical character. Check this out:
% ar rcs libnothing.a a b _c d
% ar -t libnothing.a
a
b
_c
d
I tried renaming it to aaa_syscalls.c but I have Makefile errors complaining about casio_syscalls.c not existing anymore. I have tried to make distclean (failed in the middle) and make clean before re-configuring the repo, but the error is still there. Can you help me out?
Also, how do you test whether the calloc() function used is yours? Disassembling?
However, I have some very nice hints where to look for now. I bet in the end, there is a simple solution for my problem
Keek up the good job!
Citer : Posté le 02/09/2018 14:23 | #
I'm genuinely impressed. It works! o.O
I named it aaa_casio_syscalls.c. You must not forget to edit the file name in Makefile.am and run automake (all in the same directory).
For building, make clean should do the job, but I like to keep the sources and the built files separated cleanly. That's why I have a folder build-newlib where I call ../newlib-cygwin/configure (where the sources are). All files generated by configure and make will be located in build-newlib. Before rebuilding, I simply delete build-newlib.