Updating GCC GNAT (Ada) in pkgsrc/NetBSD
Trials and tribulations of getting GCC GNAT running on NetBSD
Sitemap — — — Webring — — — Repositories — — — About
Never trust
pkgsrcwhen it says that it is installing the package. — Fernando Oleo Blanco
Abstract: This project entry is meant to document my findings about getting a modern version of GCC, in this case GCC 10, running on NetBSD with Ada support.
The author would like to acknowledge the amazing help of Jay Patelani 1, 2; the continuous comments #pkgsrc IRC community; the help of the Ada community in both Telegram1 and IRC; and finally and more important, J Marino, who created the original GCC package that supports Ada, allowed this project to take place the way it did.
The task is still unfinished as of the
Table of Contents
Current state of the art
pkgsrc currently offers two versions of GCC which support Ada. These are gcc5-aux and gcc6-aux. Both of them were created by J. Marino as part of the Aurora project, which aimed to get Ada supported in OSs that GCC supports but had no official GCC Ada port.
Recently, after pkgsrc-2021Q1 was released, pkgsrc's infrastructure regarding Ada support was updated to use gcc6-aux when building anything Ada. It was also announced that GCC 10 would be the last version of GCC that would not require a C++11 compiler, which bumps the requirements to build it. While this change of requirements for GCC is inconsequential, it is a good reason to try and get a more updated version of GCC that officially supports Ada in pkgsrc. I was also informed in #pkgsrc (pkgsrc's IRC channel2) that GCC 10 was a good starting point, since it requires fewer patches to run on NetBSD and other platforms supported by pkgsrc.
Getting it to compile
Long story short, pkgsrc allows the user to select which languages GCC will support before the building process starts by changing the options of the package. I initially added Ada to LANGS+= which should be enough to get it running. I even modified it in the options file to try to make he changes as "upstream" friendly as possible. Since that option alone did not work, I added Ada to the USE_LANGUAGES directive in the options file. However, pkgsrc does not support USE_LANGUAGES as part of the options. So after some wasted hours wondering why was Ada not getting selected, I added the ada keyword to both USE_LANGUAGES and LANGS directly in the Makefile.
Finally, Ada was getting compiled. However, in the third stage of the build3, the compilation failed since the s-osinte.ads did not comply with a set of requirements4. The issue was that the s-osinte.ads file does not really exist in GCC's source tree. That file corresponds to Ada's RTS5 and it is generated at compile time. The "generation" of the file consists of gcc/ada/Makefile.rts detecting which OS, CPU architecture pair the build is for, and populating a set of files with the appropiate contents. The Makefile.rts contains a large matching section for an incredible amount of OSs and architectures; though sadly, NetBSD is nowhere to be found.
After a lengthy discussion with the Ada community, I decided to utilize the files meant for FreeBSD as the base for NetBSD, since they are rather similar systems. This ended up being the first patch that I created in my life:
--- gcc/ada/Makefile.rtl.orig 2021-05-14 13:59:04.428016657 +0000 +++ gcc/ada/Makefile.rtl 2021-05-14 14:13:32.599013103 +0000 @@ -1739,6 +1739,37 @@ MISCLIB = -lutil endif +# x86-64 NetBSD +# For the time being, it uses the x86-64 FreeBSD config +ifeq ($(strip $(filter-out %86_64 netbsd%,$(target_cpu) $(target_os))),) + LIBGNAT_TARGET_PAIRS = \ + a-intnam.ads<libgnarl/a-intnam__freebsd.ads \ + s-inmaop.adb<libgnarl/s-inmaop__posix.adb \ + s-intman.adb<libgnarl/s-intman__posix.adb \ + s-mudido.adb<libgnarl/s-mudido__affinity.adb \ + s-osinte.adb<libgnarl/s-osinte__freebsd.adb \ + s-osinte.ads<libgnarl/s-osinte__freebsd.ads \ + s-osprim.adb<libgnat/s-osprim__posix.adb \ + s-taprop.adb<libgnarl/s-taprop__posix.adb \ + s-taspri.ads<libgnarl/s-taspri__posix.ads \ + s-tpopsp.adb<libgnarl/s-tpopsp__posix.adb \ + $(TRASYM_DWARF_UNIX_PAIRS) \ + $(ATOMICS_TARGET_PAIRS) \ + $(X86_64_TARGET_PAIRS) \ + system.ads<libgnat/system-freebsd.ads + + GNATLIB_SHARED = gnatlib-shared-dual + + EXTRA_GNATRTL_NONTASKING_OBJS += g-sse.o g-ssvety.o + EXTRA_GNATRTL_NONTASKING_OBJS += $(TRASYM_DWARF_UNIX_OBJS) + + EH_MECHANISM=-gcc + THREADSLIB= -lpthread + GMEM_LIB = gmemlib + LIBRARY_VERSION := $(LIB_VERSION) + MISCLIB = -lutil +endif + # x86-64 DragonFly ifeq ($(strip $(filter-out %86_64 dragonfly%,$(target_cpu) $(target_os))),) LIBGNAT_TARGET_PAIRS = \
The patch simply copies the FreeBSD x86_64 entry and changes the OS detection to NetBSD. After another long compilation process, GCC with Ada support finally compiled! But did it work? I created (copy-pasted) Ada's Hello World example, compiled, ran it, and got no warning, error and a very nice Hello World! output in the console!
It was time to stress-test the newly compiled gcc10-aux package. Luckly, Ada has the ACATS testing suite, and Simon Wright maintains a fork of it meant to be used with GCC. So, after some issues with ssh, requirements and PATH I got it running. After a long time and having to kill a test that was known to hang, the results were absolutely terrible. All the expected failures failed, and about half of the tests that were expected to run did not. That seemed very strange. How could a compiler compile the Hello World example, fail that many tests and still work?6
Final findings up to
After taking a look at the generated logs from ACATS, it was clear that the issue was that the linker could not find the function pthread_yield, present in the FreeBSD version of the s-osinte.ads file. pthread_yield function is not an standard pthread function, and NetBSD does not support it. After taking a look into different implementations of the Ada sched_yield function, which is the one that imported pthread_yield, it was obvious that every modern system was using the POSIX standard compliant sched_yield function; and NetBSD provides it. So, for the time being, I decided to create a nasty patch for FreeBSD's s-osinte.ads file that would replace that function. Here it is:
--- gcc/ada/libgnarl/s-osinte__freebsd.ads.orig 2021-05-18 20:30:47.702177294 +0000
+++ gcc/ada/libgnarl/s-osinte__freebsd.ads 2021-05-18 23:03:42.034134021 +0000
@@ -46,6 +46,10 @@
package System.OS_Interface is
pragma Preelaborate;
+ pragma Linker_Options ("-lrt");
+ -- Included for "sched_yield" in NetBSD
+ -- Add it before -pthread
+
pragma Linker_Options ("-pthread");
subtype int is Interfaces.C.int;
@@ -527,7 +531,7 @@
pragma Import (C, pthread_attr_getschedparam, "pthread_attr_getschedparam");
function sched_yield return int;
- pragma Import (C, sched_yield, "pthread_yield");
+ pragma Import (C, sched_yield, "sched_yield");
--------------------------
-- P1003.1c Section 16 --
After recompiling GCC with this quick and dirty patch, and running the ACATS test suite once more, another issue arose.
The tests that were previously failing, before the pthread_yield patch, were mostly related to the tasking system (RTS), which is no surprise; since s-osinte.ads is designed for that very thing. However, the new run of ACATS showed that most tests that failed before, were failing once again. After taking a look at the logs once more, no errors were found, just warnings and test failures. The failures, however, had a different pattern, some did pass after taking way too much time, others got fixed (very few), but the vast mayority were hanging. This hanging problem combined with the warnings
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__initialize': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1353: warning: warning: reference to compatibility sigaction(); include <signal.h> for correct reference /usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnat.a(s-osprim.o): in function `system__os_primitives__timed_delay': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-optide.adb:75: warning: warning: reference to compatibility nanosleep(); include <time.h> to generate correct reference /usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__monotonic__monotonic_clockXnn': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-tpopmo.adb:60: warning: warning: reference to compatibility clock_gettime(); include <time.h> to generate correct reference /usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__monotonic__rt_resolutionXnn': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-tpopmo.adb:76: warning: warning: reference to compatibility clock_getres(); include <time.h> to generate correct reference /usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__initialize': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1318: warning: warning: reference to compatibility sigaddset(); include <signal.h> for correct reference /usr/bin/ld: /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1313: warning: warning: reference to compatibility sigemptyset(); include <signal.h> for correct reference /usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnat.a(s-osprim.o): in function `system__os_primitives__clock': /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-osprim.adb:91: warning: warning: reference to compatibility gettimeofday(); include <sys/time.h> to generate correct reference
indicated that the timing primitives/structures/functions that were being used were potentially not working as expected. This leads me to believe that the clock system used in the FreeBSD s-osinte.adb implementation is not compatible with NetBSD's one.
Further work, aka, TODO
After taking a look at the Draco compiler and gcc6-aux files, specially the patches, the clock primitives and potentially some other sincronization code of the RTS has to be adapted/tweaked to work on NetBSD. These patches provide great guidance and I expect to use them to finally get Ada support in pkgsrc and NetBSD.
In the end, the idea is to have a set of files and entires (such as in the Makefile.rts system) officially added in upstream GCC, so that support would be official and be less of a burden to downstream, in this case, pkgsrc.
For the time being I find myself with no time to work on this port anymore, and I will not be able to take part of any of my other hobbies up until early September, if everything goes well.
Edit history
Fixed the link to the UnitedBSD forum. Spelling. Spelling.
Footnotes:
At the time of writing this post, I still have a Telegram account, however, I am expecting to delete it SoonTM.
I joined the channel about a month before Freenode's debacle, which in the end made #pkgsrc move to https://libera.chat
GCC has several incremental build stages.
Ada has a very expresive syntax regarding what requirements are needed in order to compile the file. In this case, a skeleton file was being selected and it was very restrictive on what could be done, since it is meant for barebone projects.
Run Time System
I have omitted that it took me an entire day to find out that all tests were failing at the beginning because I was running a remote shell connection. After that huge headache (how could GCC compile a Hello World and not pass any test?) the issue was found and solved. It just so happened that NetBSD was unable to run more =/dev/pts=s.