Skip to content

Remove the "-r" option in calling gap in the pexpect interface #27878

Open

Description

@kiwifb

Currently the pexpect interface to gap call gap with the "-r" option.
From sage/interfaces/gap.py

gap_cmd = "gap -r"

This is a remnant of gap 4.8 and under where it enabled gap to be started in a quieter mode if I am not mistaken. There is no visible effect in starting gap with or without "-r" in gap 4.10 and over.

fbissey@moonloop ~ $ gap
 ┌───────┐   GAP 4.10.1 of 23-Feb-2019
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: amd64
 Configuration:  gmp 6.1.2, readline
 Loading the library and packages ...
 Packages:   AClib 1.3.1, Alnuth 3.1.0, AtlasRep 1.5.1, AutoDoc 2019.02.22, AutPGrp 1.10, CRISP 1.4.4, Cryst 4.1.18, CrystCat 1.1.8, CTblLib 1.2.2, FactInt 1.6.2, FGA 1.4.0, 
             GAPDoc 1.6.2, IO 4.5.4, IRREDSOL 1.4, LAGUNA 3.9.2, Polenta 1.3.8, Polycyclic 2.14, PrimGrp 3.3.2, RadiRoot 2.8, ResClasses 4.7.1, SmallGrp 1.3, Sophus 1.24, 
             TomLib 1.2.7, TransGrp 2.0.4, utils 0.61
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> 
fbissey@moonloop ~ $ gap -r
 ┌───────┐   GAP 4.10.1 of 23-Feb-2019
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: amd64
 Configuration:  gmp 6.1.2, readline
 Loading the library and packages ...
 Packages:   AClib 1.3.1, Alnuth 3.1.0, AtlasRep 1.5.1, AutoDoc 2019.02.22, AutPGrp 1.10, CRISP 1.4.4, Cryst 4.1.18, CrystCat 1.1.8, CTblLib 1.2.2, FactInt 1.6.2, FGA 1.4.0, 
             GAPDoc 1.6.2, IO 4.5.4, IRREDSOL 1.4, LAGUNA 3.9.2, Polenta 1.3.8, Polycyclic 2.14, PrimGrp 3.3.2, RadiRoot 2.8, ResClasses 4.7.1, SmallGrp 1.3, Sophus 1.24, 
             TomLib 1.2.7, TransGrp 2.0.4, utils 0.61
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> 

The only effect is more subtle. ~/.gap and its equivalent on non linux system is not added to the list of paths where gap looks for packages. This means in turn that the option ignore_dot_gap in the function all_installed_packages (in sage/tests/gap_packages.py) is meaningless when a pexpect gap instance is selected. And a doctest introduced in #27681 will fail if you have something in ~/.gap.

Component: interfaces

Author: François Bissey

Branch/Commit: u/fbissey/gap_r @ 474b365

Reviewer: Steven Trogdon

Issue created by migration from https://trac.sagemath.org/ticket/27878

Activity

  1. added this to the sage-8.8 milestone on May 27, 2019
  2. kiwifb commented on May 28, 2019

    @kiwifb
    Author
  3. kiwifb commented on May 28, 2019

    @kiwifb
    MemberAuthor

    Commit: 8958e4d

  4. kiwifb commented on May 28, 2019

    @kiwifb
    MemberAuthor
  5. kiwifb commented on May 28, 2019

    @kiwifb
    MemberAuthor
    comment:2

    Trivial change.


    New commits:

    8958e4dremove extraneous -r from gap_cmd
  6. kiwifb commented on May 28, 2019

    @kiwifb
    MemberAuthor

    Author: François Bissey

  7. strogdon commented on May 28, 2019

    @strogdon
    Contributor

    Reviewer: Steven Trogdon

  8. strogdon commented on May 28, 2019

    @strogdon
    Contributor
    comment:3

    Here I have

    $ ls -R ~/.gap
    /home/volj/27/strogdon/.gap:
    pkg
    
    /home/volj/27/strogdon/.gap/pkg:
    AtlasRep
    
    /home/volj/27/strogdon/.gap/pkg/AtlasRep:
    datagens  dataword
    
    /home/volj/27/strogdon/.gap/pkg/AtlasRep/datagens:
    
    /home/volj/27/strogdon/.gap/pkg/AtlasRep/dataword:
    

    The doctest ./sage -t --long src/sage/tests/gap_packages.py, with this branch, passes which would fail without the fix.

    Hopefully the positive review will prompt someone to notice if there is any side effect.

  9. dimpase commented on May 28, 2019

    @dimpase
    Member
    comment:4
    $ gap -h
    ...
     -r                           disable/enable user GAP root dir
                                  GAPInfo.UserGapRoot
    

    also

      The  root  directories  are  specified via one or several of the -l paths command line options, see 3.1. Furthermore, by default GAP automatically prepends a user
      specific GAP root directory to the list; this can be avoided by calling GAP with the -r option. The name of this user specific directory depends on your operating
      system, it can be found in GAPInfo.UserGapRoot. This directory can be used to tell GAP about personal preferences, to always load some additional code, to install
      additional packages, or to overwrite some GAP files. See 3.2 for more information how to do this.
    
  10. 32 remaining items

  11. embray commented on Jun 7, 2019

    @embray
    Contributor
    comment:30

    Replying to @kiwifb:

    I am trying to answer too fast. I see you are trying to avoid negative settings and I can understand that. I guess we just have to bear the negative use internally.

    Yeah; the sense of the command-line parameter is negative, so this can be documented internally; but that doesn't mean the user-interface in Sage needs to be as well :)

  12. embray commented on Jun 7, 2019

    @embray
    Contributor
    comment:31

    Replying to @kiwifb:

    Replying to @embray:

    Replying to @nthiery:

    Feedback from Alex and Michael: it used to be that GAP package authors could control whether their package was loaded automatically, and many if not most did. Nowadays, the normal behavior is not to autoload (like in e.g. Python or R), unless GAP core or some other autoloaded package requires it. So most recent packages don't autoload. But many old packages still autoload, for backward compatibility.

    Yep--in particular autoloading of the xgap package has been a real problem for Sage's pexpect interface.

    Now that's a painful flashback from before you started being involved with sage Erik. It took me a while to figure out what was happening to my sage-on-gentoo install where I had installed the whole gap distribution.

    I'll concede that your point (C) sounds reasonable but has the after taste of the package you customise specifically for sage (or some other software).

    Off course we want to move all of the use of point (C) to libgap I would imagine , which leaves the pexpect interface at your point (B). But having upstream giving you the tools to get to (C) would be nice.

    Indeed it would be nice; but it's admittedly not easy. I've been watching the efforts over the last few years to make multiple parallel embedded Python interpreters possible from libpython, and while it mostly works now it was not a trivial effort. I think it's maybe a little easier for GAP than it was for Python, especially with HPC-GAP, where most of the important interpreter globals have been moved into thread-local variables. But it's still non-trivial, especially when you involve things like C extension modules :/

    It's something I would gladly help work on if I had the time to do so, which I don't (and I doubt most of the GAP developers do either, even if it would likely be a desirable goal).

  13. embray commented on Jun 7, 2019

    @embray
    Contributor
    comment:32

    The other complication to this that I haven't addressed yet is that GAP should probably also load a different workspace (if any) for when it's running with or without the .gap directory loaded, or this could lead to even more confusing inconsistencies :/

    Perhaps the arguments passed when starting GAP should be hashed into the hash in the workspace filename, or something? I'm not confident about this part as I still don't fully understand how GAP workspaces work (just enough to know that they can definitely be influenced heavily by what pakages and globals are loaded).

  14. embray commented on Jun 7, 2019

    @embray
    Contributor
    comment:33

    Replying to @embray:

    Principle of less surprise: keep GAP in Sage as similar as possible to GAP outside of Sage.

    I think that the gap global made available to users in Sage should not be the same as that used internally by Sage. I think a lot of problems like this can be resolved cleanly by not crossing these use-cases where it isn't necessary to.

    I see now, after experimenting with this, that there is a technical challenge to it (I still think it is the right approach in principle). The problem is that although it is possible to have multiple Gap interpreter instances, much of the code in Sage makes inconsistent assumptions that you are only using the "main" instance, currently the global variable named gap.

    For example, SageObjects can have both a _gap_ method, and a _gap_init_ method. The former returns the object's GAP representation wrapped in a GapElement. It can take an optional Gap instance as an argument: The GAP interpreter instance in which to create the element. The latter, _gap_init_, only returns a string of GAP code to be evaluated in the GAP interpreter to instantiate the object. This is consistent with how other interfaces work.

    The problem (or one of them) is that _gap_init_ does not take an optional Gap instance, and just always assumes you're using the "main" gap instance. This can lead to all kinds of inconsistencies, as sometimes the _gap_init_ methods themselves query the Gap instance (e.g. what is the current $sage variable number) for details on how to evaluate the object. So if you use one Gap instance in the _gap_init_ call, but then pass the result of that call to a different Gap instance it is not guaranteed to work.

    For this reason, among others, I would propose that we keep the current status quo w.r.t. disabling .gap by default for now, but add the enable_dotgap option, so that it's easy to create a new Gap instance that uses it. Then change the default behavior only once further refactoring is done to maintain better separation between the (B) and (C) use cases.

  15. kiwifb commented on Jun 10, 2019

    @kiwifb
    MemberAuthor
    comment:34

    All right. I think it is also important to have the doctest that started it all to be fixed quickly.

  16. nthiery commented on Jun 11, 2019

    @nthiery
    Contributor
    comment:35

    Replying to @embray:

    For libgap we're in a bit of a bind, but I think that the most important use case for the libgap interface in Sage is for interfacing with GAP internally (to replace the pexpect interface as much as possible), so this should disable .gap by default.

    In my research code (and in packages such as sage-semigroups), I really need to be using libgap (rather than a pexpect interface) for efficiency. I believe that's a common use case.

    I see the rationale for aiming two separate gap instances, an incorruptible one for internal use, and a more flexible one for direct access by users. I don't know whether that's technically feasible at this stage though with libgap. Also we still want to be able to pass gap objects efficiently from internal code to user code.

  17. dimpase commented on Jun 11, 2019

    @dimpase
    Member
    comment:36

    the sooneer a complete switch to libgap happens in sagelib, the better.

    I don't quite understand what "pass gap objects efficiently from internal code to user code" means here. IMHO this all fine in libgap, no?

  18. nthiery commented on Jun 11, 2019

    @nthiery
    Contributor
    comment:37

    the sooneer a complete switch to libgap happens in sagelib, the better.

    Yes, indeed!

    I don't quite understand what "pass gap objects efficiently from internal code to user code" means here. IMHO this all fine in libgap, no?

    Currently yes, indeed. Now assume we were to have two separate instances of libgap, one for sagelib and one user-facing, to protect the former from corruption by e.g. user packages; then, it may become costly to move one object from one instance to the other.

  19. dimpase commented on Jun 11, 2019

    @dimpase
    Member
    comment:38

    I'm not sure many instances of libgap loaded as python extensions are technically feasible; this seem to mean that one would have to mangle names in the dlls to avoid clashes etc.

    What might be feasible are several libgap workspaces.

  20. embray commented on Jun 14, 2019

    @embray
    Contributor
    comment:39

    Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).

  21. modified the milestones: sage-8.8, sage-8.9 on Jun 14, 2019
  22. kiwifb commented on Jul 23, 2019

    @kiwifb
    MemberAuthor
    comment:40

    I am opening a new ticket to fix the doctest in gap_packages under all scenario #28233. Let's keep this ticket for the serious stuff about how we are supposed to run gap.

  23. removed this from the sage-8.9 milestone on Dec 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions