NPTL (Native POSIX Threading Library) Tests and Trace

01/17

Our server has changed, which should result in better speed of this web site.

12/20

The final report for the second Test Campaign, as well as the project, is available in the Papers section of the site. Direct link

11/22

A few changes to this website, for a clearer presentation (I hope). All documents (including OLS material) are now located in the Papers section.

11/16

A new test campaign is on its way. The results will be added to the results database. It will serve as a base to fix some remaining problems both in nptl and in the test suite. More details will come later.

The forum is closed now. No new message can be posted; but old messages can still be read.

07/13

The schedule for the Ottawa Linux Symposium has been posted! (Here). As you can see, we'll be giving our presentation on Saturday the 23rd, at noon. Come and see us ;)

06/16

Here is an extract from Fedora Core 4 release notes:

" As the next step in removing support for the obsolete LinuxThreads library,
 code compiled and linked on Fedora Core 4 now
 automatically uses the NPTL headers and libraries.

In previous releases since Red Hat Linux 9, the default threading was LinuxThreads
 since the interface is mostly forward compatible. 
The advantages of the NPTL interface is that the cancellation handling is faster 
(when -fexception is used, even in C code) and that the additional 
interfaces are now available without special compiler and linker parameters. 
In other words, you do not need to use
 -I/usr/include/nptl and -L/usr/lib{,64}/nptl
 any longer. Note that lib{,64} is to be interpreted as lib64 
on platforms which place DSOs in lib64/ directories, 
and lib/ otherwise. "

06/15

New PTT released! This new release brings some new features and adds support for IA64 and PPC architectures. See project homepage for more information.

05/20

ptt-beta1 was released last week. This is a beta release, meaning almost all functionnalities are added and the interface should not change a lot from now on. Have a look at the tool webpage for more information and documentation.

04/13

Long time since previous news... In the meantime, the website has been updated to include links to the PTT project (was previously known as NPTL Trace project). See links section for more information.

The PTT release alpha3 is out. See this URL for release notes and downloads. Thanks to contributors!

03/09

The latest testcase for sem_unlink has just been written. So, test writting for P2 function list is complete.

03/04

Four functions, related to Thread-Local Data (TLD) have been downgraded from P2 to P3 list: pthread_key_create, pthread_getspecific, pthread_setspecific and pthread_key_delete. These functions are not used really often in Linux, as gcc provides the __thread keyword which offers better performances and simpler use.

As the sem_unlink analysis has just been written, the P2-routines list is now completed. In a few days, all tests should be written and completed, and the final test campaign can start.

03/03

The news just felt: We'll give a presentation on our NPTL Test & Trace project at the Ottawa Linux Symposium. See this web site for more information on the symposium.

03/01

The forum subscription has been disabled temporarly because of an exploit in phpBB allowing any user to log in as administrator. No new user accepted until this issue is fixed.

Actually the issue is already solved in phpBB 2.0.13, the forum has been updated, subscription is reopened.

02/28

Two new problems have been found in the NPTL. The first one is a conformance problem: sem_open() returns a wrong error code when there is no space left. This has been reported as bug 774 in bugzilla. The second one is a scalability issue with sem_close. See the graphical example. This has not been reported, as explained in the Results section.

02/23

We've passed the 10.000 hits on this site, a few days ago! (and the forum's hits do not count into this). The statistics (not available to public) show that most people come from a google site...

As for the progress, the thread cancelation functions are almost done. Remaining are the semaphores and the thread-local data functions, before priority 2 completion.

02/22

With the end of the sigwait function, the 200th testcase has been written and submitted to OPTS. Cheers!

02/21

The previously reported bug 749 was actually a false one: the problem was in the testcase, not in the implementation. This is now fixed.

Good news: nothing to test for the pthread_cleanup_push and pthread_cleanup_pop routines, which lets us pass the 50% of P2 milestone on time :-).

02/17

A third bug has been opened in bugzilla for the sigaction routine: bug 749.

02/16

Long time no bug had been reported... Well, this is changing :).

Two new bugs have been opened for the sigaction routine: Bug 746 and Bug 748.

02/15

The sourceforce CVS access has been restored finally, after 4 days.

Analysis for the pthread_cancel routine is now available in the forum.

02/14

The conformance test for the pthread_sigmask has been written. See detail in the forum.

Also, the analysis for the sigaction function is terminated. The OPTS already contains 400 testcases for this routine, but some of the assertions are not tested yet... Lot of work to come!

And by the way... Happy Valentine's Day ;)

02/10

New testcases for the pthread_kill() routine have been written and submitted to the OPTS. Find more...

There is currently an issue with SourceForge.net services preventing the ViewCVS and the anonymous CVS services to work.

02/04

The sigprocmask routine has been analyzed. No testing needed for this one :).

02/03

New tests for the pthread_atfork have been written and submitted to the Open POSIX Test Suite. See here for details.

02/01

3 new conformance tests have been written for the pthread_setschedparam routine. See detail.

A new official glibc package has been released recently (2.3.4). The trace patch will be updated to this official release in a near future.

01/25

The sched_yield() routine has been analyzed and appears not to need any new test case.

01/24

The pthread_getschedparam function has now been covered. See the forum for more information.

01/21

tslogparser v04 is out. This is mainly an updated documentation and a new RPM packaging.

01/19

The TSLogParser interface for Test Campaign 1 (on this website) has been updated to v03. New link to these results added in the menu ('database').

01/18

3 conformance tests have been added to Open POSIX Test Suite for the pthread_join routine. See detail in this forum thread.

With this new addition, the total of tests written is now 100. You can see a graphical progression of the writting here (as you can see, I'm not familiar to GD).

01/17

4 conformance tests and 1 stress test have been added to Open POSIX Test Suite for the pthread_once routine. See detail in this forum thread.

01/11

The TSLogParser version 0.3 has just been released. See the project homepage for more information.

The testcases for the fork() routine have been released to OPTS. See this post for details. 18 assertions are covered out of 24, by 19 testcases.

BTW, the latest submitted glibc bug (problem in unistd.h) has been fixed very quickly...

01/06

A new bug has been raised into bugzilla. It concerns the header which does not make visible some symbols. See bug report for more details.

01/05

Happy new year!

Some code has been put into the nptltracetool CVS repository. This may be discussed on the project mailing list (see http://sf.net/projects/nptltracetool for details).

12/20

The NPTL Trace Tool project now has a separate place located on SourceForge.Net. See the project summary page for more information and downloads (nothing available yet).

12/17

The Open POSIX Test Suite 1.5.0 has just been released. See http://posixtest.sourceforge.net/ for more information.

12/14

Tests for pthread_self() routine are now written.

Analysis for pthread_once() and pthread_join() routine is now in the forum.

12/01

The test for the first function of P2 list is now written. More details can be found here.

11/30

The analysis for the first three routines have been written and can be found there in the forum. The fork() routine counts 24 assertions, and is currently missing from OPTS... Huge work to do here!

11/29

It's been a while since the latest testcase was released. It is going to change, as we're starting to analyse the priority-2 routines. The procedure will be the same as for priority-1: List the POSIX assertions, Check the existing glibc and OPTS testcases for what assertions are already tested, and finally write new testcases for the remaining assertions, when it is possible. Any contributor is -still- welcome to help analysing or test writting...

11/25

The tslogparser project has its own website now: http://tslogparser.sourceforge.net.

The priority 2 list of routines for the testing effort has been updated. Two API are added, pthread_getschedparam() and pthread_setschedparam(), to match our users requirements. The full list in available in this forum's thread.

11/23

The testsuite runs browser interface has become a separate project, hosted on sourceforge: http://sourceforge.net/projects/tslogparser/.

11/19

<quote>
Hello Everybody,
The Issue related to the SCHED_RR threads bug has been now reported to the OSDL Kernel Bug Tracker:
http://bugzilla.kernel.org/
It is Bug: #3770
Cheers,
Loic.
</quote>

11/18

The final report for the first Test Campaign is on-line. You can find it in the "download" section, with a summary of the results.

The results browser is still under development, but some additionnal results may be added to the database for Solaris runs soon.

11/16

The web interface to test campaign results is now in beta-version here. It allows to browse a specific run, or to compare 2 or more runs and show only the differences (for example, compare Fedora Core 3 glibc with Suse 9 one, or vanilla glibc on i686 and Itanium2, etc...). Don't forget this is a beta version ;) You won't be able to add new information, this is normal. If you find something like a bug, please let me know ;). This engine might become part of the STP (OSDL) website later, so the graphical design won't be refined yet.

11/08

A new web interface is being written to allow tests results parsing. Test campaign results will be made available through this interface, so the publication is postponed a little. The need for this interface comes from the constatation that 10 runs x 1846 testcases each is too much data for manual analysis...

10/28

The Test Campaign is almost terminated; the results will be processed and made available soon. Stay tuned for more information!

10/18

A defect has been opened in RedHat bugzilla, thanks to Loïc, for the scheduler issue which has been previously discussed here. More details can be found there.

10/15

A new section has been added in the website: 'Download'. It currently contains only links to other web resources, but the Trace code will be available there when it is ready.

10/13

The stress test for the pthread_mutex_lock() routine indeed *had* a design flaw. It has now been fixed.

10/12

The 20th user in the forum has just registered :-) Welcome!

Some draft documentation on the NPTL Trace Tool Project (driven by French University contributors) has been added to the website. It can be found here. More documentation and source code will follow in the near future. This material is opened to discussion in the forum.

While starting the test campaign, some unexpected (?) errors have appeared:

• The stress test for the pthread_mutex_lock routine crashes with a segmentation fault. Looking quickly at the source code showed no design flaw.
• SLES 9 on ppc64 is somewhat tricky to upgrade: compiling a newer binutils package fails, compiling a newer vanilla kernel fails, make check in the glibc fails, running a compiled sample linked against the new glibc produces strange errors.... This case will be removed from the test plans!

10/08

The October LTP has been released. It contains quite a lot of cleanups and fixes in testcases and in Makefile.

09/30

A new bug has been opened in bugzilla: bug #417. Thanks to Alexander from comp.programming.threads for putting this to my attention.

09/28

The conformance test for the pthread_equal() routine has been written and posted to OPTS. Detail can be found here.

The conformance test for the pthread_detach() routine have been written and posted to OPTS. Detail can be found here.

This marks the end of the priority-1 function testing (15 routines).

• 107 assertions from POSIX were examined.
• 91 (84%) are now fully tested in Open POSIX TestSuite -- the others cannot be tested.
• 63 (75%) of those were partially or completly contributed by our project
• 14 additionnal assertions were tested with scalability and stress tests.
• A total of 70 testcases were written.
• 54 conformance tests
• 6 scalability tests
• 10 stress tests
• 12 bugs were found in the NPTL. 7 bug reports were opened in BugZilla.
• Stress and conformance tests showed 6 bugs each.
• No scalability issue has been found yet.

09/27

Bug #405 has been fixed in glibc-cvs. The pthread_exit() finally can execute without error, and therefor has been added to Open POSIX Test Suite. See this thread for details.

09/24

The bug #401 has been fixed in CVS, according to Bugzilla.

At this point, only the bug 300 seems to be remaining for the NPTL. The discussions indicated that this one is related to gcc.

Oops erratum: I just opened a new bug: bug 405 in bugzilla. This is a segmentation fault in pthread_create() under some circumstances. Still working on this pthread_exit() stress test...

As far as the scheduler bug is concerned (see SCHED_RR discussions in the forum and the mailing list archive), a new version of the sample test is being worked on by Loïc and should be published soon.

09/23

A new bug has been posted bugzilla BZ401. This concerns a memory leak in pthread_create and was found thanks to the pthread_exit() stress test.

The pthread_exit() conformance tests (7 tests) have been written and added to Open POSIX TestSuite, as usual.

09/22

The analysis of the pthread_detach() routine is now in the forum. This routine closes the #1 priority function list. Once the tests will be written, a large test campaign on several architectures will follow, previously to the #2 priority testing. More details on this test campaign will be published later.

09/21

The stress test for the pthread_mutex_trylock() routine is now posted to Open POSIX TestSuite.

Analysis for the pthread_equal() routine is now in the forum. Only one assertion need additional test.

The conformance tests for the pthread_mutex_destroy() routine have been written and added to Open POSIX TestSuite.

09/20

According to glibc CVS logs, the bug #386 has been fixed, but the status in bugzilla did not change yet.

Analysis for the pthread_exit() routine is now done. 7 assertions need more testing. This means that the priority-#1 function testing will terminate probably around mid-october (previous estimation was end september).

The conformance tests for the pthread_mutex_trylock() routine have been written and posted to OPTS. No new problem was found with those tests.

09/17

A new scalability test has been written for the pthread_create() routine. A sample graphical output can be found in the forum here.

The scheduler issue (SCHED_RR) is not closed yet. Looks like there is a bug in 2.6.x kernel scheduler. Further testing is being done.

09/15

A new bug has been opened concerning the pthread_create() routine (# 386). This one is minor as it concerns a bad return value of the function in case of error.

The Results page has been updated with the addition of the new conformance tests for the pthread_create() routine.

09/14

The conformance tests for the pthread_create() routine are now written and have been added to the Open POSIX Test Suite. The detail is in the forum.

Concerning the Ballista project, it seems that the project is no longer maintened, so new tests won't be added.

09/13

Bugzilla' bugs #320 (rwlock #define dependances) and #379 (thread executes on pthread_create failure) have been solved in glibc CVS.

09/10

The bug #347 has been reposted as bug #379 as the description did not fit anymore with the bug content.

I just found another testsuite for POSIX routines: The Ballista POSIX Testsuite. This testsuite include some tests for the sem_* routines, but none for the pthread_* routines...

09/09

Found a possible explanation to bug #347. The problems seems not to be what I saw at first; the hanging futex being the trace mutex' one. Actually it looks like when pthread_create() fails, the not-created thread can execute some instructions before not being created ;). This is due to a possible late delivery of the cancel signal. A new sample, with additionnal explanation, has been posted into bugzilla (347). The "results" page has been updated accordingly.

09/08

Many thanks to Loic for his great support on the scheduling issues. Not solved yet, but it is on its way.

Thanks also to Bryce for his XML parser of assertions.xml files from PTS, which will enable to get assertion text in STP execution reports.

09/03

The bug #342 has been fixed in libc' CVS.

The STP interface to PTS is now working. You can find an exemple of output here: http://khack.osdl.org/stp/297188/. There are some bugs remaining bug the squeletton is now OK.

08/31

I've had some hard time dealing with the scheduler attributes... Well, I'm giving up! If people want to help you can find information on the problem in the comp.programming.threads newsgroup archive and in the forum. I tested the test algorithm with Linux, Solaris and Aix5L and none is behaving as I expected so...

08/26

Yet another bug, this one in the pthread_create() routine. The detail can be found in bugzilla bug #347. The problem appears when a thread creation fails then another call to pthread_create() make the application hang.

08/24

New coverage analysis for the pthread_mutex_destroy() routine is now in the forum. 2 assertions need more testing.

08/23

A new non-conformance to POSIX has been found in NPTL, which results in a hang in applications which are conformant to POSIX. This bug is in the pthread_cond_destroy routine, which can succeed and return while the condvar is still used in other routines. This has been reported in BugZilla bug #342. You may find additionnal information there.

08/20

Analysis for the pthread_mutex_trylock() routine is available in the forum. 5 assertions need more testing.

08/19

Three new conformance tests have been written for the pthread_cond_broadcast() routine. Here is the detail.

With this new addition, more than 50% of our #1 priority routines are covered.

08/17

Analysis for the pthread_create() routine is now available in the forum. There is a large amount of work to achieve on this function -- contributors still wanted ^_^.

Here is a summary of the current forum situation:

• 6 functions have been completed so far (raising 7 bugs in NPTL).
• 3 additionnal functions have been analysed, and specification for the tests to be written are available. The functions are:
  • pthread_cond_broadcast(). Tests are currently being written for this function.
  • pthread_cond_destroy().
  • and pthread_create().

08/16

Analysis for the pthread_cond_destroy() routine is now available in the forum.

Conformance test for the pthread_cond_signal() routine is now written and posted to PTS. More detail available here.

08/13

All three defect reports which were sent to the aardvark for the POSIX standard were accepted:

• XSH ERN 43 pthread_mutex_lock Accept
• XSH ERN 44 pthread_cond_timedwait EINVAL Error should be described consistently with other interfaces Accept
  For the error EINVAL,
   change: "The value specified by abstime is invalid."
   with: "The abstime argument specified a nanosecond value less than zero or greater than or equal to 1000 million."
• XSH ERN 50 pthread_cond_wait with a recursive mutex that is locked several time. Accept as marked below
  The standard does not speak to this issue, and as such no conformance distinction can be made between alternative implementations based on this. This permits implementations to choose how to handle situations where an application might deadlock, some implementations might choose to detect and report that deadlock while others wouldn't.

New stress test for the pthread_cond_wait() and pthread_cond_timedwait() routines. This checks that no condition signaling is being lost. Details in the forum.

08/12

All conformance tests for the pthread_cond_wait() routine are now written and integrated in PTS. Detail can be found here.

08/11

A new bug was reported to Bugzilla: #320. The problem is that the read-write lock features are unavailable to a program which claims strict POSIX conformance (by defining only _POSIX_C_SOURCE), whereas these features are part of the POSIX spec.

The "Results" page has been updated. It now contains a progress indicator for this project.

08/09

Analysis for the pthread_cond_broadcast() routine is now available in the forum.

A new Request For Clarification has been submitted to the OpenGroup. This concerns the pthread_cond_wait/pthread_cond_timedwait routines. The detail can be found here (request #50).

08/05

Concerning the pthread_cond_timedwait/stress2 issue, a bug was just reported to the glibc bugzilla (Bug #300). This sample spotted actually two different problems for the function pthread_cancel(), which are described in the Results page.

The LTP has announced the august release. This new release contains bugfixes and some new security tests.

08/04

A new project has been created into OSDL dedicated to the inclusion of the Open POSIX Test Suite into the Scalable Test Platform, as a new test suite. More details on this project here.

08/02

The new website is now working. The old page can still be found here.

07/29

The design of this website has been remanied. This section News appears.

The two reports for POSIX problems, Enhancement Request Number 43 and Enhancement Request Number 44 were accepted and are now published into the Aardvark defect reports (XSH is concerned).

I just finished a timeout utility. You can find the source here in the forum. Next step is to use this tool inside the PTS to detect the hanging testcases and report the hangs.

07/28

A new project has been proposed in OSDL for the inclusion of the Open POSIX Testsuite into the Scalable Test Platform project.

The STP is a platform which "provides a framework where developers can test patches to different kinds of software against a set of performance and scalability tests on various hardware platforms. The process of requesting a test run is straight forward and offered through a web interface."

The long-term goal is to have a database of POSIX-compliance information with several distributions / kernels. To achieve this goal, more work has to be done on this interface, so people will become used to use it.
A first step will be to generate complete reports and a list of failed POSIX assertions, for a given platform. When people have an easy way to see what is wrong with a given platform, improving the platform will become a straightforward process.
Another good use of this database will be for people porting user applications. The porting will become easier when you can see what POSIX features are supported or not on your target platform.

07/27

Order for the remaining NPTL routines analysis (and tests) can be found and discussed in this forum's thread. Comments are welcome.

07/21

I just gave a try to gcov, to cover the NPTL routines usage. The problem is, gcc is unable to add coverage tracking inside the assembler source files; so I'd have to start again for an IA64 NPTL (whole source is in C). Anyway, the results of the PTS coverage (i686) test can be found here.

07/07

July release of the LTP. Many improvements and bug fixes. The embedded PTS is 1.4.3 (with bugfix).

07/06

The Open POSIX Test Suite 1.4.3 has just been released. This release includes the scalability and stress tests which have been written here so far.

07/01

Two stress tests for the pthread_cond_timedwait() have been written and can be found here in the forum. It seems that stress2.c has spotted a problem but I need to investigate more on this... after the vacation ;-).

06/29

Evolution of all the existing scalability tests. Each tests now include support for the PLOT_OUTPUT macro, and mathematical support to match the routine behavior with a model, and then decide whether the function is scalable or not. The returned values have been upgraded accordingly. Screenshots of graphical output can be found in each forum threads.

Stress and scalability tests are now included into the PTS.

06/22

Scalability test for the pthread_cond_timedwait() routine is here.

This sample has a support for graphical output, through the macro PLOT_OUTPUT. When compiled with this macro defined, the test will output some data which can be passed to the do-plot script (same location as above). This script will do some formatting and then used gnuplot to build a graph (or a png file, for example).

06/11

Six conformance tests for the pthread_cond_timedwait() routine have been written. Details, as usual, are in the forum.

06/10

You can find a new scalability test for the pthread_cond_init() routine here.

06/08

June release of the LTP. Contains new testcases and bugfixes.

06/03

Bug fix in the stress test for pthread_mutex_lock() (array overflow problem).

A new stress test for the pthread_cond_init() routine is here.

05/27

Two conformance tests written for the routine pthread_cond_init().
The testfrmw.* files have also been updated.

Open POSIX Test Suite 1.4.2 has just been released. This is the first release which includes some tests from our project.

05/24

The two tests for the pthread_mutex_unlock() routine are now included into the PTS.

05/19

Two conformance tests were written for the pthread_mutex_unlock() routine. Detail in this forum's thread

05/14

Remaining tests for the pthread_mutex_init() routine, i.e.

• speculative/5-2
• 5-1

05/12

First tests have been integrated to the PTS.

Those tests are conformance tests for the pthread_mutex_init() routine. Three testcases are concerned.

New stress test for the pthread_mutex_lock() routine. Details in the forum (this thread).

05/04

In order to ease integration of the tests into the PTS, some return codes and the naming convention of the tests has changed. See details inside the forum.

Three conformance tests were written for the pthread_mutex_lock() routine. Source code and explanations can be found in the forum.

05/03

The Open POSIX Test Suite 1.4.1 has been released. Website: http://posixtest.sourceforge.net.

04/30

Tests design evolution.

A new file, 'testfrmw.c', has been added in the design. This file contains the "output()" function. It allows to change the output format (or destination) of every tests in this project, without editing every source files. This file will be included in every testcase archives.

A new stress test has been written for the routine pthread_mutex_init(). The code and the description can be found in the forum thread.

04/29

First submission to the Open POSIX Test Suite project mailing-list (posixtest-discuss_at_lists.sourceforge.net).

Three conformance testcases were submitted. The source and description of those tests can be found in the forum.

The project's forum is now set up. The email support is broken, but the forum allows file uploading, which is very usefull.

03/30

Coverage analysis of NPTL is now terminated. The testing work can now begin...

02/10

First bug spotted in NPTL (I'm a lucky guy), within pthread_barrier_wait . See Results section for details.

01/12

Start of this project -- well, actually, some preliminary work has already been done !