pexpect interfaces are never deleted #17686

sagetrac-alexc · 2015-01-29T00:16:09Z

This crashes because the Dokchitser instances are kept alive all the time.

sage: for i in range(2000):
....:     D = Dokchitser(conductor=1, gammaV=[0], weight=1, eps=1, poles=[1], residues=[-1], init='1')

Component: interfaces

Author: Jeroen Demeyer

Branch: 15e42fe

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/17686

The text was updated successfully, but these errors were encountered:

jdemeyer · 2015-01-29T06:31:57Z

Replying to @sagetrac-alexc:

Let me know what additional information I can provide to help.

A reproducible example...

sagetrac-alexc · 2015-01-30T05:28:52Z

Attachment: computeltest.py.gz

Example

sagetrac-alexc · 2015-01-30T05:30:28Z

comment:2

Replying to @jdemeyer:

Replying to @sagetrac-alexc:

Let me know what additional information I can provide to help.

A reproducible example...

I've attached a program which produces the error. The program fails after count = 160 for me.

jdemeyer · 2015-01-31T09:45:33Z

Dependencies: #17704

jdemeyer · 2015-02-02T22:02:44Z

New commits:

`8501bb1`	`Clean up in expect.py`
`821151d`	`Implement proper kill-on-del for pexpect`

jdemeyer · 2015-02-02T22:02:44Z

Branch: u/jdemeyer/ticket/17686

jdemeyer · 2015-02-02T22:02:44Z

Author: Jeroen Demeyer

jdemeyer · 2015-02-02T22:02:44Z

Commit: 821151d

sagetrac-git · 2015-02-03T11:31:32Z

Changed commit from 821151d to 012c62c

sagetrac-git · 2015-02-03T11:31:32Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`d556229`	`Further clean-up of interfaces`
`012c62c`	`Implement proper kill-on-del for pexpect`

jdemeyer · 2015-02-03T11:31:57Z

Changed dependencies from #17704 to #17704, #17718

rwst · 2015-02-25T16:20:37Z

comment:10

With #17718 merged:

sage -t --long src/sage/interfaces/sagespawn.pyx  # 2 doctests failed
sage -t --long src/sage/interfaces/quit.py  # 1 doctest failed

sagetrac-git · 2015-02-25T17:13:02Z

Changed commit from 012c62c to fede9b0

sagetrac-git · 2015-02-25T17:13:02Z

Branch pushed to git repo; I updated commit sha1. New commits:

`c8d1a28`	`Revert interrupt change`
`815fbc6`	`Merge commit 'c8d1a289e95184f3f6f5528e8fb8ee4638ab47c0' into t/17686/ticket/17686`
`47646e9`	`Merge remote-tracking branch 'origin/develop' into t/17686/ticket/17686`
`fede9b0`	`Fix doctest for GP`

jdemeyer · 2015-02-25T17:14:20Z

comment:12

Merged #17718 and latest develop. I also trivially fixed the quit.py failure but I don't get any failures in sagespawn.pyx. Can you try again and post the details of the failure if they still occur?

sagetrac-git · 2015-05-27T08:21:46Z

Changed commit from 864d55a to 4810d72

jdemeyer · 2015-05-27T08:22:02Z

Changed dependencies from #17704, #17718 to none

sagetrac-git · 2015-05-29T20:05:05Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`08e422a`	`Implement proper kill-on-del for pexpect`

sagetrac-git · 2015-05-29T20:05:05Z

Changed commit from 4810d72 to 08e422a

vbraun · 2015-06-17T12:03:21Z

comment:27

The original spawn class checks self.closed in the dtor:

    def __del__(self):
        if self.closed:
            return
        try:
            self.close()
        except:
            pass

So why not just set that flag in _keep_alive? Or is there another resource managed by an attribute with a non-trivial dtor? This seems better than keeping the spawn instance alive forever... maybe I'm missing something?

jdemeyer · 2015-06-17T12:29:47Z

comment:28

Replying to @vbraun:

So why not just set that flag in _keep_alive? Or is there another resource managed by an attribute with a non-trivial dtor? This seems better than keeping the spawn instance alive forever... maybe I'm missing something?

Py_INCREF(self) makes no assumptions on how pexpect.spawn works internally. Setting self.closed would work currently, but it would for example break with a newer version of pexpect.

vbraun · 2015-06-17T17:36:52Z

Reviewer: Volker Braun

vbraun · 2015-06-17T20:08:42Z

comment:30

Fails on OSX:

sage -t --long src/sage/interfaces/expect.py  # 26 doctests failed
sage -t --long src/sage/interfaces/gap.py  # 95 doctests failed
sage -t --long src/sage/interfaces/mwrank.py  # 6 doctests failed

sagetrac-git · 2015-06-18T08:49:50Z

Branch pushed to git repo; I updated commit sha1. New commits:

`15e42fe`	`Wrap sendline() in try/except`

sagetrac-git · 2015-06-18T08:49:50Z

Changed commit from 08e422a to 15e42fe

vbraun · 2015-06-18T20:27:34Z

comment:34

I got this (with #17924, only randomly fails)

sage -t --long src/sage/interfaces/sagespawn.pyx
**********************************************************************
File "src/sage/interfaces/sagespawn.pyx", line 125, in sage.interfaces.sagespawn.SageSpawn.close
Failed example:
    while s.isalive():  # long time (5 seconds)
        sleep(0.1)
Exception raised:
    Traceback (most recent call last):
      File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.sagespawn.SageSpawn.close[3]>", line 1, in <module>
        while s.isalive():  # long time (5 seconds)
      File "/home/buildslave-sage/slave/sage_git/build/local/lib/python2.7/site-packages/pexpect.py", line 762, in isalive
        pid, status = os.waitpid(self.pid, waitpid_options)
      File "sage/ext/interrupt/interrupt.pyx", line 197, in sage.ext.interrupt.interrupt.sage_python_check_interrupt (/home/buildslave-sage/slave/sage_git/build/src/build/cythonized/sage/ext/interrupt/interrupt.c:1743)
        sig_check()
      File "sage/ext/interrupt/interrupt.pyx", line 86, in sage.ext.interrupt.interrupt.sig_raise_exception (/home/buildslave-sage/slave/sage_git/build/src/build/cythonized/sage/ext/interrupt/interrupt.c:884)
        raise KeyboardInterrupt
    KeyboardInterrupt
**********************************************************************
1 item had failures:
   1 of   5 in sage.interfaces.sagespawn.SageSpawn.close
    [24 tests, 1 failure, 5.08 s]

jdemeyer · 2015-06-19T08:03:32Z

comment:35

Replying to @vbraun:

I got this (with #17924, only randomly fails)

On which machine?

This is probably yet another race condition: it looks like we are killing the process group of the child process when it has not yet created a new process group. So, we end up killing ourselves.

vbraun · 2015-06-19T08:24:31Z

comment:36

That was on arando. I'm going to close this ticket and leave the remaining race to you, then ;-)

vbraun · 2015-06-19T08:25:05Z

Changed branch from u/jdemeyer/ticket/17686 to 15e42fe

vbraun · 2015-06-19T22:24:21Z

Changed commit from 15e42fe to none

vbraun · 2015-06-19T22:24:21Z

comment:38

Followup at #18741

sagetrac-alexc mannequin added this to the sage-6.5 milestone Jan 29, 2015

sagetrac-alexc mannequin added c: number theory labels Jan 29, 2015

This comment has been minimized.

Sign in to view

jdemeyer added c: interfaces and removed c: number theory labels Jan 30, 2015

jdemeyer changed the title ~~computel.gp: "unable to start pari because the command 'gp --emacs --quiet --stacksize 10000000' failed"~~ pexpect interfaces don't close logfile Jan 30, 2015

This comment has been minimized.

Sign in to view

jdemeyer changed the title ~~pexpect interfaces don't close logfile~~ pexpect interfaces are never deleted Jan 31, 2015

jdemeyer added the s: needs review label Feb 2, 2015

rwst modified the milestones: sage-6.5, sage-6.6 Feb 25, 2015

rwst added s: needs work and removed s: needs review labels Feb 25, 2015

vbraun added s: positive review and removed s: needs review labels Jun 17, 2015

vbraun added s: needs work and removed s: positive review labels Jun 17, 2015

jdemeyer added s: needs review and removed s: needs work labels Jun 18, 2015

vbraun added s: positive review and removed s: needs review labels Jun 18, 2015

vbraun removed the s: positive review label Jun 19, 2015

vbraun closed this as completed in 17a174f Jun 19, 2015

jdemeyer mentioned this issue Feb 18, 2015

Small clean-up in expect.py #17704

Closed

sagetrac-Vincent-Neri mannequin mentioned this issue Jun 19, 2015

Race condition in pexpect #17924

Closed

This was referenced Mar 30, 2017

Race condition in pexpect - part 2 #18721

Closed

Remove a lot of deprecated code #24103

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pexpect interfaces are never deleted #17686

pexpect interfaces are never deleted #17686

sagetrac-alexc mannequin commented Jan 29, 2015

jdemeyer commented Jan 29, 2015

sagetrac-alexc mannequin commented Jan 30, 2015

sagetrac-alexc mannequin commented Jan 30, 2015

This comment has been minimized.

This comment has been minimized.

jdemeyer commented Jan 31, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

sagetrac-git mannequin commented Feb 3, 2015

sagetrac-git mannequin commented Feb 3, 2015

jdemeyer commented Feb 3, 2015

rwst commented Feb 25, 2015

sagetrac-git mannequin commented Feb 25, 2015

sagetrac-git mannequin commented Feb 25, 2015

jdemeyer commented Feb 25, 2015

sagetrac-git mannequin commented May 27, 2015

jdemeyer commented May 27, 2015

sagetrac-git mannequin commented May 29, 2015

sagetrac-git mannequin commented May 29, 2015

vbraun commented Jun 17, 2015

jdemeyer commented Jun 17, 2015

vbraun commented Jun 17, 2015

vbraun commented Jun 17, 2015

sagetrac-git mannequin commented Jun 18, 2015

sagetrac-git mannequin commented Jun 18, 2015

vbraun commented Jun 18, 2015

jdemeyer commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015

pexpect interfaces are never deleted #17686

pexpect interfaces are never deleted #17686

Comments

sagetrac-alexc mannequin commented Jan 29, 2015

jdemeyer commented Jan 29, 2015

sagetrac-alexc mannequin commented Jan 30, 2015

sagetrac-alexc mannequin commented Jan 30, 2015

This comment has been minimized.

This comment has been minimized.

jdemeyer commented Jan 31, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

jdemeyer commented Feb 2, 2015

sagetrac-git mannequin commented Feb 3, 2015

sagetrac-git mannequin commented Feb 3, 2015

jdemeyer commented Feb 3, 2015

rwst commented Feb 25, 2015

sagetrac-git mannequin commented Feb 25, 2015

sagetrac-git mannequin commented Feb 25, 2015

jdemeyer commented Feb 25, 2015

sagetrac-git mannequin commented May 27, 2015

jdemeyer commented May 27, 2015

sagetrac-git mannequin commented May 29, 2015

sagetrac-git mannequin commented May 29, 2015

vbraun commented Jun 17, 2015

jdemeyer commented Jun 17, 2015

vbraun commented Jun 17, 2015

vbraun commented Jun 17, 2015

sagetrac-git mannequin commented Jun 18, 2015

sagetrac-git mannequin commented Jun 18, 2015

vbraun commented Jun 18, 2015

jdemeyer commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015

vbraun commented Jun 19, 2015