Updating Ada support in NetBSD

Never trust pkgsrc when it says it is installing the package.
—Fernando Oleo Blanco

Current state of the art

pkgsrc currently offers two versions of GCC which support Ada. These are gcc5-aux and gcc6-aux. Both of them were created by J. Marino as part of the Aurora project, which aimed to get Ada supported in OSs that GCC supports but had no official GCC Ada port.

Recently, after pkgsrc-2021Q1 was released, pkgsrc’s infrastructure regarding Ada support was updated to use gcc6-aux when building anything Ada. It was also announced that GCC 10 would be the last version of GCC that would not require a C++11 compiler, which bumps the requirements to build it. While this change of requirements for GCC is inconsequential, it is a good reason to try and get a more updated version of GCC that officially supports Ada in pkgsrc. I was also informed in #pkgsrc (pkgsrc’s IRC channel1) that GCC 10 was a good starting point, since it requires fewer patches to run on NetBSD and other platforms supported by pkgsrc.

Getting it to compile

Long story short, pkgsrc allows the user to select which languages GCC will support before the building process starts by changing the options of the package. I initially added Ada to LANGS+= which should be enough to get it running. I even modified it in the options file to try to make he changes as “upstream” friendly as possible. Since that option alone did not work, I added Ada to the USE_LANGUAGES directive in the options file. However, pkgsrc does not support USE_LANGUAGES as part of the options. So after some wasted hours wondering why was Ada not getting selected, I added the ada keyword to both USE_LANGUAGES and LANGS directly in the Makefile.

Finally, Ada was getting compiled. However, in the third stage of the build2, the compilation failed since the s-osinte.ads did not comply with a set of requirements3. The issue was that the s-osinte.ads file does not really exist in GCC’s source tree. That file corresponds to Ada’s RTS4 and it is generated at compile time. The “generation” of the file consists of gcc/ada/Makefile.rts detecting which OS, CPU architecture pair the build is for, and populating a set of files with the appropiate contents. The Makefile.rts contains a large matching section for an incredible amount of OSs and architectures; though sadly, NetBSD is nowhere to be found.

After a lengthy discussion with the Ada community, I decided to utilize the files meant for FreeBSD as the base for NetBSD, since they are rather similar systems. This ended up being the first patch that I created in my life:

--- gcc/ada/Makefile.rtl.orig   2021-05-14 13:59:04.428016657 +0000
+++ gcc/ada/Makefile.rtl    2021-05-14 14:13:32.599013103 +0000
@@ -1739,6 +1739,37 @@
   MISCLIB = -lutil
 endif

+# x86-64 NetBSD
+# For the time being, it uses the x86-64 FreeBSD config
+ifeq ($(strip $(filter-out %86_64 netbsd%,$(target_cpu) $(target_os))),)
+  LIBGNAT_TARGET_PAIRS = \
+  a-intnam.ads<libgnarl/a-intnam__freebsd.ads \
+  s-inmaop.adb<libgnarl/s-inmaop__posix.adb \
+  s-intman.adb<libgnarl/s-intman__posix.adb \
+  s-mudido.adb<libgnarl/s-mudido__affinity.adb \
+  s-osinte.adb<libgnarl/s-osinte__freebsd.adb \
+  s-osinte.ads<libgnarl/s-osinte__freebsd.ads \
+  s-osprim.adb<libgnat/s-osprim__posix.adb \
+  s-taprop.adb<libgnarl/s-taprop__posix.adb \
+  s-taspri.ads<libgnarl/s-taspri__posix.ads \
+  s-tpopsp.adb<libgnarl/s-tpopsp__posix.adb \
+  $(TRASYM_DWARF_UNIX_PAIRS) \
+  $(ATOMICS_TARGET_PAIRS) \
+  $(X86_64_TARGET_PAIRS) \
+  system.ads<libgnat/system-freebsd.ads
+
+  GNATLIB_SHARED = gnatlib-shared-dual
+
+  EXTRA_GNATRTL_NONTASKING_OBJS += g-sse.o g-ssvety.o
+  EXTRA_GNATRTL_NONTASKING_OBJS += $(TRASYM_DWARF_UNIX_OBJS)
+
+  EH_MECHANISM=-gcc
+  THREADSLIB= -lpthread
+  GMEM_LIB = gmemlib
+  LIBRARY_VERSION := $(LIB_VERSION)
+  MISCLIB = -lutil
+endif
+
 # x86-64 DragonFly
 ifeq ($(strip $(filter-out %86_64 dragonfly%,$(target_cpu) $(target_os))),)
   LIBGNAT_TARGET_PAIRS = \

The patch simply copies the FreeBSD x86_64 entry and changes the OS detection to NetBSD. After another long compilation process, GCC with Ada support finally compiled! But did it work? I created (copy-pasted) Ada’s Hello World example, compiled, ran it, and got no warning, error and a very nice Hello World! output in the console!

It was time to stress-test the newly compiled gcc10-aux package. Luckly, Ada has the ACATS testing suite, and Simon Wright maintains a fork of it meant to be used with GCC. So, after some issues with ssh, requirements and PATH I got it running. After a long time and having to kill a test that was known to hang, the results were absolutely terrible. All the expected failures failed, and about half of the tests that were expected to run did not. That seemed very strange. How could a compiler compile the Hello World example, fail that many tests and still work?5

Final findings up to 2021-05-27

After taking a look at the generated logs from ACATS, it was clear that the issue was that the linker could not find the function pthread_yield, present in the FreeBSD version of the s-osinte.ads file. pthread_yield function is not an standard pthread function, and NetBSD does not support it. After taking a look into different implementations of the Ada sched_yield function, which is the one that imported pthread_yield, it was obvious that every modern system was using the POSIX standard compliant sched_yield function; and NetBSD provides it. So, for the time being, I decided to create a nasty patch for FreeBSD’s s-osinte.ads file that would replace that function. Here it is:

--- gcc/ada/libgnarl/s-osinte__freebsd.ads.orig  2021-05-18 20:30:47.702177294 +0000
+++ gcc/ada/libgnarl/s-osinte__freebsd.ads       2021-05-18 23:03:42.034134021 +0000
@@ -46,6 +46,10 @@
 package System.OS_Interface is
    pragma Preelaborate;

+   pragma Linker_Options ("-lrt");
+   --  Included for "sched_yield" in NetBSD
+   --  Add it before -pthread
+
    pragma Linker_Options ("-pthread");

    subtype int            is Interfaces.C.int;
@@ -527,7 +531,7 @@
    pragma Import (C, pthread_attr_getschedparam, "pthread_attr_getschedparam");

    function sched_yield return int;
-   pragma Import (C, sched_yield, "pthread_yield");
+   pragma Import (C, sched_yield, "sched_yield");

    --------------------------
    -- P1003.1c  Section 16 --

After recompiling GCC with this quick and dirty patch, and running the ACATS test suite once more, another issue arose.

The tests that were previously failing, before the pthread_yield patch, were mostly related to the tasking system (RTS), which is no surprise; since s-osinte.ads is designed for that very thing. However, the new run of ACATS showed that most tests that failed before, were failing once again. After taking a look at the logs once more, no errors were found, just warnings and test failures. The failures, however, had a different pattern, some did pass after taking way too much time, others got fixed (very few), but the vast mayority were hanging. This hanging problem combined with the warnings

/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__initialize':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1353: warning: warning: reference to compatibility sigaction(); include <signal.h> for correct reference
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnat.a(s-osprim.o): in function `system__os_primitives__timed_delay':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-optide.adb:75: warning: warning: reference to compatibility nanosleep(); include <time.h> to generate correct reference
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__monotonic__monotonic_clockXnn':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-tpopmo.adb:60: warning: warning: reference to compatibility clock_gettime(); include <time.h> to generate correct reference
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__monotonic__rt_resolutionXnn':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-tpopmo.adb:76: warning: warning: reference to compatibility clock_getres(); include <time.h> to generate correct reference
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnarl.a(s-taprop.o): in function `system__task_primitives__operations__initialize':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1318: warning: warning: reference to compatibility sigaddset(); include <signal.h> for correct reference
/usr/bin/ld: /usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-taprop.adb:1313: warning: warning: reference to compatibility sigemptyset(); include <signal.h> for correct reference
/usr/bin/ld: /home/fernando/mysandboxfolder/usr/pkg/gcc10/lib/gcc/x86_64--netbsd/10.3.0/adalib/libgnat.a(s-osprim.o): in function `system__os_primitives__clock':
/usr/pkgsrc/wip/gcc10-aux/work/build/gcc/ada/rts/s-osprim.adb:91: warning: warning: reference to compatibility gettimeofday(); include <sys/time.h> to generate correct reference

indicated that the timing primitives/structures/functions that were being used were potentially not working as expected. This leads me to believe that the clock system used in the FreeBSD s-osinte.adb implementation is not compatible with NetBSD’s one.

Further work as of 2021-05-28

After taking a look at the Draco compiler and gcc6-aux files, specially the patches, the clock primitives and potentially some other sincronization code of the RTS has to be adapted/tweaked to work on NetBSD. These patches provide great guidance and I expect to use them to finally get Ada support in pkgsrc and NetBSD. In the end, the idea is to have a set of files and entires (such as in the Makefile.rts system) officially added in upstream GCC, so that support would be official and be less of a burden to downstream, in this case, pkgsrc.

For the time being I find myself with no time to work on this port anymore, and I will not be able to take part of any of my other hobbies up until early September, if everything goes well.

Finishing up the work

Well… It turns out that the pthread structures defined in the RTS where no longer valid for NetBSD. This happened because NetBSD introduced some changes to its internal data structures, and such changes, had to be refelcted within Ada’s data imports. This issue was found and fixed by Marino, who I owe the finalisation of my initial work. I do not really know how he found that the underliying pthread and related data structures had changed, but his experience does really shine here.

The fix was to read the underlying structures in the adaint.h file and export them to be visible by the Ada code in order to import them. Once the patches were applied… GNAT ran pretty much the entire testsuite to completion!! What an acomplishment! You can find the work done by Marino in his set of patches in the Draco repository.

An extra bit

Now that we managed to make GNAT run on a modern NetBSD system, I wanted to test how far the newly built GNAT could go… So I tried cross-compiling it for other NetBSD architectures. I will not explain my trial and tribulations in detail. But I can tell you that it was painful to learn the entire canadian build process, getting the correct versions of all the dependencies (I am looking at you binutils) and then manually fix every compilation error that I found. However, after all this pain, I managed to get GCC with Ada support cross-compiled for NetBSD PowerPC. I copied the files to a virtual machine, uploaded the ACATS testing suite and ran it… Boy was it slow, but little by little, I could see most tests pass without many complaints! What an achievement!

After all this work, I can proudly say that GCC and Ada rock. I learnt a ton from this project and I had the wonderful help from some of the most knowledgeable programmers I have met. I think, without much doubt, that this journey enabled me to deal with complex systems and projects like no other taks I had done up until now.

Edit history

2021-05-28 Fixed the link to the UnitedBSD forum. Spelling.

2021-08-14 Spelling.

2024-03-28 Well… I finally finished writing the article, sight…

Footnotes

Footnotes

  1. I joined the channel about a month before Freenode’s debacle, which in the end made #pkgsrc move to Libera Chat ↩

  2. GCC has several incremental build stages. ↩

  3. Ada has a very expresive syntax regarding what requirements are needed in order to compile the file. In this case, a skeleton file was being selected and it was very restrictive on what could be done, since it is meant for barebone projects. ↩

  4. Run Time System ↩

  5. I have omitted that it took me an entire day to find out that all tests were failing at the beginning because I was running a remote shell connection. After that huge headache (how could GCC compile a Hello World and not pass any test?) the issue was found and solved. It just so happened that NetBSD was unable to run more /dev/pts. ↩