[RFC] watchthreads2.exp: gdb can drop watchpoints

[dje at google dot com (Doug Evans)]

Hi.

I wrote this testcase to help exercise h/w watchpoints in linux gdbserver.
[A patch I'm working on, based on Pierre's win32 patch.]

Running it with just gdb periodically fails.
I traced this to gdb dropping watchpoints if, for example,
an shlib breakpoint from a different thread happens at the same time
(e.g. one happens and then while gdb is stopping the remaining threads
the other happens?).

One can see this from the output in gdb.log:

[...]
(gdb) continue
Continuing.
Thread 0 changing x 11 -> 12
Thread 0 changing x 12 -> 13
[Switching to Thread 0xf7e56bb0 (LWP 7892)]
Hardware watchpoint 3: x

Old value = 12
New value = 13
0x080483c3 in thread_function (arg=0x0) at ../../../../src/gdb/testsuite/gdb.threads/watchthreads2.c:110
110	      ++x; usleep (1);  /* X increment.  */
(gdb) FAIL: gdb.threads/watchthreads2.exp: x watch loop
[...]

Note that gdb missed the 11 -> 12 case above.

Turning on "set debug infrun 1" we see this.
Here gdb missed the 14 -> 15 case.

[...]
Hardware watchpoint 3: x

Old value = 13
New value = 14
0x080483c3 in thread_function (arg=0x3) at ../../../../src/gdb/testsuite/gdb.threads/watchthreads2.c:110
110	      ++x; usleep (1);  /* X increment.  */
(gdb) continue
Continuing.
infrun: clear_proceed_status_thread (Thread 0xf6563bb0 (LWP 22743))
infrun: clear_proceed_status_thread (Thread 0xf6db4bb0 (LWP 22742))
infrun: clear_proceed_status_thread (Thread 0xf7605bb0 (LWP 22741))
infrun: clear_proceed_status_thread (Thread 0xf7e56bb0 (LWP 22740))
infrun: clear_proceed_status_thread (Thread 0xf7e576b0 (LWP 22737))
infrun: proceed (addr=0xffffffff, signal=144, step=0)
infrun: resume (step=0, signal=0), trap_expected=0
Thread 3 changing x 14 -> 15
infrun: wait_for_inferior (treat_exec_as_sigtrap=0)
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7ff3101
infrun: stopped by watchpoint
infrun: stopped data address = 0x8049af8
infrun: context switch
infrun: Switching context from Thread 0xf6563bb0 (LWP 22743) to Thread 0xf6db4bb0 (LWP 22742)
infrun: BPSTAT_WHAT_CHECK_SHLIBS
infrun: no stepping, continue
infrun: resume (step=1, signal=0), trap_expected=1
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7ff3102
infrun: no stepping, continue
infrun: resume (step=0, signal=0), trap_expected=0
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6563bb0 (LWP 22743)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x80483c3
infrun: context switch
infrun: Switching context from Thread 0xf6db4bb0 (LWP 22742) to Thread 0xf6563bb0 (LWP 22743)
infrun: no stepping, continue
infrun: resume (step=0, signal=0), trap_expected=0
Thread 3 changing x 15 -> 16
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7ff3101
infrun: stopped by watchpoint
infrun: stopped data address = 0x8049af8
infrun: context switch
infrun: Switching context from Thread 0xf6563bb0 (LWP 22743) to Thread 0xf6db4bb0 (LWP 22742)
infrun: BPSTAT_WHAT_CHECK_SHLIBS
infrun: no stepping, continue
infrun: resume (step=1, signal=0), trap_expected=1
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7ff3102
infrun: no stepping, continue
infrun: resume (step=0, signal=0), trap_expected=0
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7fb3735
infrun: BPSTAT_WHAT_SINGLE
infrun: no stepping, continue
infrun: resume (step=1, signal=0), trap_expected=1
infrun: prepare_to_wait
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6db4bb0 (LWP 22742)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0xf7fb3736
infrun: no stepping, continue
infrun: resume (step=0, signal=0), trap_expected=0
infrun: prepare_to_wait
[Thread 0xf6db4bb0 (LWP 22742) exited]
infrun: target_wait (-1, status) =
infrun:   22737 [Thread 0xf6563bb0 (LWP 22743)],
infrun:   status->kind = stopped, signal = SIGTRAP
infrun: infwait_normal_state
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x80483c3
infrun: context switch
infrun: Switching context from Thread 0xf6db4bb0 (LWP 22742) to Thread 0xf6563bb0 (LWP 22743)
infrun: BPSTAT_WHAT_STOP_NOISY
infrun: stop_stepping
Hardware watchpoint 3: x

Old value = 15
New value = 16
0x080483c3 in thread_function (arg=0x3) at ../../../../src/gdb/testsuite/gdb.threads/watchthreads2.c:110
110	      ++x; usleep (1);  /* X increment.  */
(gdb) FAIL: gdb.threads/watchthreads2.exp: x watch loop
[...]


I don't know if you want to wait until the bug is fixed before this
testcase can be checked in.  I won't be able to look at the bug
for awhile but maybe someone else will, so I'm submitting this testcase.
It (periodically) fails on i386-linux, amd64-linux (depending on
how the o/s happened to schedule the threads of course).

2009-04-29  Doug Evans  <dje@google.com>

	* gdb.threads/watchthreads2.exp: New testcase.
	* gdb.threads/watchthreads2.c: New testcase.

Index: watchthreads2.c
===================================================================
RCS file: watchthreads2.c
diff -N watchthreads2.c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ watchthreads2.c	29 Apr 2009 22:58:14 -0000
@@ -0,0 +1,115 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2009 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+   Check that watchpoints get propagated to all existing threads when the
+   watchpoint is created.
+*/
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <pthread.h>
+
+#ifndef NR_THREADS
+#define NR_THREADS 4
+#endif
+
+#ifndef X_INCR_COUNT
+#define X_INCR_COUNT 10
+#endif
+
+void *thread_function (void *arg); /* Function executed by each thread.  */
+
+pthread_mutex_t x_mutex;
+int x;
+
+/* Used to hold threads back until watchthreads2.exp is ready.  */
+int test_ready = 0;
+
+int
+main ()
+{
+  int res;
+  pthread_t threads[NR_THREADS];
+  int i;
+
+  pthread_mutex_init (&x_mutex, NULL);
+
+  for (i = 0; i < NR_THREADS; ++i)
+    {
+      res = pthread_create (&threads[i],
+			    NULL,
+			    thread_function,
+			    (void *) (intptr_t) i);
+      if (res != 0)
+	{
+	  fprintf (stderr, "error in thread %d create\n", i);
+	  abort ();
+	}
+    }
+
+  for (i = 0; i < NR_THREADS; ++i)
+    {
+      res = pthread_join (threads[i], NULL);
+      if (res != 0)
+	{
+	  fprintf (stderr, "error in thread %d join\n", i);
+	  abort ();
+	}
+    }
+
+  exit (EXIT_SUCCESS);
+}
+
+/* Easy place for a breakpoint.
+   watchthreads2.exp uses this to track when all threads are running
+   instead of, for example, the program keeping track
+   because we don't need the program to know when all threads are running,
+   instead we need gdb to know when all threads are running.
+   There is a delay between when a thread has started and when the thread
+   has been registered with gdb.  */
+
+void
+thread_started ()
+{
+}
+
+void *
+thread_function (void *arg)
+{
+  int i;
+
+  thread_started ();
+
+  /* Don't start incrementing X until watchthreads2.exp is ready.  */
+  while (! test_ready)
+    usleep (1);
+
+  for (i = 0; i < X_INCR_COUNT; ++i)
+    {
+      pthread_mutex_lock (&x_mutex);
+      /* For debugging.  */
+      printf ("Thread %ld changing x %d -> %d\n", (long) arg, x, x + 1);
+      /* The call to usleep is so that when the watchpoint triggers,
+	 the pc is still on the same line.  */
+      ++x; usleep (1);  /* X increment.  */
+      pthread_mutex_unlock (&x_mutex);
+    }
+
+  pthread_exit (NULL);
+}
Index: watchthreads2.exp
===================================================================
RCS file: watchthreads2.exp
diff -N watchthreads2.exp
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ watchthreads2.exp	29 Apr 2009 22:58:14 -0000
@@ -0,0 +1,125 @@
+# This testcase is part of GDB, the GNU debugger.
+
+# Copyright 2009 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that watchpoints get propagated to all existing threads when the
+# watchpoint is created.
+
+set NR_THREADS 4
+set X_INCR_COUNT 10
+
+if $tracelevel {
+    strace $tracelevel
+}
+
+set prms_id 0
+set bug_id 0
+
+# This test verifies that a watchpoint is detected in the proper thread
+# so the test is only meaningful on a system with hardware watchpoints.
+if [target_info exists gdb,no_hardware_watchpoints] {
+    return 0;
+}
+
+set testfile "watchthreads2"
+set srcfile ${testfile}.c
+set binfile ${objdir}/${subdir}/${testfile}
+if {[gdb_compile_pthreads "${srcdir}/${subdir}/${srcfile}" "${binfile}" executable [list debug "incdir=${objdir}" "additional_flags=-DNR_THREADS=$NR_THREADS -DX_INCR_COUNT=$X_INCR_COUNT"]] != "" } {
+    return -1
+}
+
+gdb_exit
+gdb_start
+gdb_reinitialize_dir $srcdir/$subdir
+gdb_load ${binfile}
+
+gdb_test "set can-use-hw-watchpoints 1" "" ""
+
+#
+# Run to `main' where we begin our tests.
+#
+
+if ![runto_main] then {
+    gdb_suppress_tests
+}
+
+gdb_test "break thread_started" \
+         "Breakpoint 2 at .*: file .*${srcfile}, line .*" \
+         "breakpoint on thread_started"
+
+# Run the program until all threads have hit thread_started.
+# We use this as the vehicle to determine when gdb is aware
+# of all threads.
+
+for { set i 0 } { $i < $NR_THREADS } { incr i } {
+    gdb_test "continue" \
+	".*Breakpoint 2, thread_started ().*" \
+	"run to thread_started"
+}
+
+# Watch X, it will be modified by all threads.
+# We want this watchpoint to be set *after* all threads are running.
+gdb_test "watch x" "Hardware watchpoint 3: x"
+
+# Now that the watchpoint is set, we can let the threads increment X.
+gdb_test "set var test_ready = 1" ""
+
+# While debugging.
+#gdb_test "set debug infrun 1" ""
+
+set x_inc_line [gdb_get_line_number "X increment"]
+set x_thread_loc "thread_function \\\(arg=.*\\\) at .*watchthreads.c:$x_inc_line"
+
+# X is incremented under a mutex, so we should get NR_THREADS * X_INCR_COUNT
+# hits.
+set limit [expr $NR_THREADS*$X_INCR_COUNT]
+set x_count 0
+set done 0
+
+set message "x watch loop"
+
+for {set i 0} {!$done && $i < $limit} {incr i} {
+    set test_flag 0
+
+    gdb_test_multiple "continue" $message {
+	-re "(.*Hardware watchpoint.*)$gdb_prompt $" {
+	    # At least one hardware watchpoint was hit.  Check if both were.
+	    set string $expect_out(1,string)
+
+	    if [regexp "Hardware watchpoint 3: x\[^\r\]*\r\[^\r\]*\r\[^\r\]*Old value = $x_count\[^\r\]*\r\[^\r\]*New value = [expr $x_count+1]\r" $string] {
+		incr x_count
+		set test_flag 1
+	    } else {
+		fail $message
+	    }
+	}
+	-re "The program is not being run.*$gdb_prompt $" {
+	    fail "$message (program terminated)"
+	}
+    }
+
+    # If we fail above, don't bother continuing loop.
+    if { $test_flag == 0 } {
+	set done 1
+    }
+}
+
+set message "all threads incremented x"
+if { $i == $limit } {
+    pass $message
+} else {
+    fail $message
+}