Re: [PATCH] Make "backtrace" doesn't print python stack if init python dir get fail

[Hui Zhu <hui_zhu at mentor dot com>]

On 12/04/13 04:26, Tom Tromey wrote:
> > > > > > "Hui" == Hui Zhu <hui_zhu@mentor.com> writes:
>
> Hui> I got a issue is ensure_python_env check gdb_python_initialized and
> Hui> throw error if it is 0.  So gdb_python_initialized to 0 will make
> Hui> some commands throw error when python dir has something error.  I
> Hui> removed this check in the patch.
>
> I think that's a useful consistency check, so better left in place.
>
> Two other approaches are possible here instead.  One, change
> finish_python_initialization to do the needed bit of locking by handy,
> not using ensure_python_env.  Or, two, don't release the GIL until
> somewhere in finish_python_initialization, and then it doesn't need
> to call ensure_python_env at all.
>
> Tom

Hi Tom,

I think the first way is better than second way because ensure_python_env has relationship with current_arch and current_language.
So I make a patch according the first way.

It extends gdb_python_initialized to a flags.  And add GDB_PYTHON_INITIALIZED_BASE and GDB_PYTHON_INITIALIZED_DIR as its flags.
In _initialize_python, set GDB_PYTHON_INITIALIZED_BASE to gdb_python_initialized.
In gdb_python_initialized, plus GDB_PYTHON_INITIALIZED_DIR to gdb_python_initialized.
Then in the function that need python lib support (apply_frame_filter, start_type_printers), check GDB_PYTHON_INITIALIZED_DIR.

It handle the issue and pass the regression test in x86_64 linux.

Please help me review it.

Thanks,
Hui

2013-12-04  Hui Zhu  <hui@codesourcery.com>

	* python/py-framefilter.c (apply_frame_filter): Change the check
	of "gdb_python_initialized".
	* python/python-internal.h (GDB_PYTHON_INITIALIZED_BASE,
	GDB_PYTHON_INITIALIZED_DIR): New.
	(gdb_python_initialized): Add comments.
	* python/python.c (gdb_python_initialized): Remove comments.
	(ensure_python_env): Change the check of "gdb_python_initialized".
	(start_type_printers): Ditto.
	(_initialize_python): Change the setup of "gdb_python_initialized".
	(finish_python_initialization): Add setup of
	"gdb_python_initialized".

--- a/gdb/python/py-framefilter.c
+++ b/gdb/python/py-framefilter.c
@@ -1474,7 +1474,7 @@ apply_frame_filter (struct frame_info *f
   PyObject *item;
   htab_t levels_printed;
 
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_DIR) == 0)
     return PY_BT_NO_FILTERS;
 
   cleanups = ensure_python_env (gdbarch, current_language);
--- a/gdb/python/python-internal.h
+++ b/gdb/python/python-internal.h
@@ -204,6 +204,14 @@ struct program_space;
 struct bpstats;
 struct inferior;
 
+/* This bit will be true if Python libary has been successfully
+   initialized.  */
+#define GDB_PYTHON_INITIALIZED_BASE	0x1
+/* This bit will be true if Python directory has been successfully
+   initialized.  */
+#define GDB_PYTHON_INITIALIZED_DIR	0x2
+
+/* The status of Python initialized.  */
 extern int gdb_python_initialized;
 
 extern PyObject *gdb_module;
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -73,9 +73,6 @@ static const char *gdbpy_should_print_st
 #include "interps.h"
 #include "event-top.h"
 
-/* True if Python has been successfully initialized, false
-   otherwise.  */
-
 int gdb_python_initialized;
 
 static PyMethodDef GdbMethods[];
@@ -151,7 +148,7 @@ ensure_python_env (struct gdbarch *gdbar
   struct python_env *env = xmalloc (sizeof *env);
 
   /* We should not ever enter Python unless initialized.  */
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_BASE) == 0)
     error (_("Python not initialized"));
 
   env->state = PyGILState_Ensure ();
@@ -1235,7 +1232,7 @@ start_type_printers (void)
   struct cleanup *cleanups;
   PyObject *type_module, *func = NULL, *result_obj = NULL;
 
-  if (!gdb_python_initialized)
+  if ((gdb_python_initialized & GDB_PYTHON_INITIALIZED_DIR) == 0)
     return NULL;
 
   cleanups = ensure_python_env (get_current_arch (), current_language);
@@ -1717,7 +1714,7 @@ message == an error message without a st
 
   make_final_cleanup (finalize_python, NULL);
 
-  gdb_python_initialized = 1;
+  gdb_python_initialized = GDB_PYTHON_INITIALIZED_BASE;
   return;
 
  fail:
@@ -1807,6 +1804,8 @@ finish_python_initialization (void)
      variable.  */
 
   do_cleanups (cleanup);
+
+  gdb_python_initialized |= GDB_PYTHON_INITIALIZED_DIR;
   return;
 
  fail: