[conda] Cloning env SOMETIMES downloads everything again/--offline ignored

[Ian Sudbery]

Hi All,

I'm posting here for advice rather than github as an issue because I can't
manage to get a minimal reproducible example.

Does anyone have any idea what might cause conda to download packages
afresh when cloning? I am trying to clone an env, but everytime a I do it
tries to redownload all the packages. This is a problem because that is
15Gb of stuff, and its not going to take many clones before this becomes a
monster.

This is not because of missing tar balls
(https://github.com/conda/conda/issues/7398) because I'm trying to do this
immediately after installing the env I'm trying to clone. And in any case
the package cache is still full of all the package tarballs.

This does not happen for every env: I can build a really simple env and
clone it no problem:

$ conda create -n test bzip2
Solving environment: done

## Package Plan ##

environment location:
/shared/sudlab1/General/apps/conda/conda-install/envs/test

added / updated specs:
- bzip2


The following NEW packages will be INSTALLED:

bzip2: 1.0.6-h470a237_2 conda-forge
libgcc-ng: 7.2.0-hdf63c60_3 conda-forge

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > source activate test
#
# To deactivate an active environment, use:
# > source deactivate
#


$ conda create -n test2 --clone test
Source: /shared/sudlab1/General/apps/conda/conda-install/envs/test
Destination: /shared/sudlab1/General/apps/conda/conda-install/envs/test2
Packages: 2
Files: 0
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > source activate test2
#
# To deactivate an active environment, use:
# > source deactivate
#

I've even tried, with the `--offline` option to create, but it still
downloads (I would have thought it would at least give an error if it can't
complete with --offline rather than just using online anyway)

$ conda create -n mb1ims_sharc --clone cgat-f --offline
Source: /shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f
Destination:
/shared/sudlab1/General/apps/conda/conda-install/envs/mb1ims_sharc
Packages: 498
Files: 189

Downloading and Extracting Packages
ca-certificates-2018 |
###########################################################################################################################################################################################################################################
| 100%
krb5-1.14.6 |
##################################################################################################################################################################################4
| 76%


Here is my `conda info`

$ conda info -a

active environment : cgat-f
active env location :
/shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f
shell level : 1
user config file : /home/mb1ims/.condarc
populated config files :
conda version : 4.5.11
conda-build version : not installed
python version : 3.7.0.final.0
base environment : /shared/sudlab1/General/apps/conda/conda-install
(writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/linux-64
https://repo.anaconda.com/pkgs/pro/noarch
package cache : /shared/sudlab1/General/apps/conda/conda-install/pkgs
/home/mb1ims/.conda/pkgs
envs directories : /shared/sudlab1/General/apps/conda/conda-install/envs
/home/mb1ims/.conda/envs
platform : linux-64
user-agent : conda/4.5.11 requests/2.19.1 CPython/3.7.0
Linux/3.10.0-862.14.4.el7.x86_64 centos/7 glibc/2.17
UID:GID : 28002:5300
netrc file : None
offline mode : False

# conda environments:
#
alexia /home/mb1ims/.conda/envs/alexia
sequana-test /home/mb1ims/.conda/envs/sequana-test
sharc /home/mb1ims/.conda/envs/sharc
sudlab /home/mb1ims/.conda/envs/sudlab
base /shared/sudlab1/General/apps/conda/conda-install
cgat-f * /shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f
macs2 /shared/sudlab1/General/apps/conda/conda-install/envs/macs2
sicer /shared/sudlab1/General/apps/conda/conda-install/envs/sicer
splicing /shared/sudlab1/General/apps/conda/conda-install/envs/splicing
test /shared/sudlab1/General/apps/conda/conda-install/envs/test
test2 /shared/sudlab1/General/apps/conda/conda-install/envs/test2
tophat2 /shared/sudlab1/General/apps/conda/conda-install/envs/tophat2
/tmp/0
/tmp/1

sys.version: 3.7.0 | packaged by conda-forge | (defau...
sys.prefix: /shared/sudlab1/General/apps/conda/conda-install
sys.executable: /shared/sudlab1/General/apps/conda/conda-install/bin/python
conda location:
/shared/sudlab1/General/apps/conda/conda-install/lib/python3.7/site-packages/conda
conda-build: None
conda-env: /shared/sudlab1/General/apps/conda/conda-install/bin/conda-env
user site dirs:

CIO_TEST: <not set>
CONDA_BACKUP_JAVA_HOME:
CONDA_BACKUP_JAVA_LD_LIBRARY_PATH:
CONDA_DEFAULT_ENV: cgat-f
CONDA_EXE: /shared/sudlab1/General/apps/conda/conda-install/bin/conda
CONDA_PREFIX: /shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f
CONDA_PROMPT_MODIFIER: (cgat-f)
CONDA_PYTHON_EXE:
/shared/sudlab1/General/apps/conda/conda-install/bin/python
CONDA_ROOT: /shared/sudlab1/General/apps/conda/conda-install
CONDA_SHLVL: 1
DRMAA_LIBRARY_PATH: /usr/local/sge/live/lib/lx-amd64/libdrmaa.so.1.0
JAVA_LD_LIBRARY_PATH:
/shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f/jre/lib/amd64/server
MANPATH:
/usr/local/sge/live/man:/usr/share/man:/usr/local/share/man:/usr/X11R6/man:/usr/share/man:/usr/local/share/man
MODULEPATH:
/shared/sudlab1/General/modulefiles:/shared/sudlab1/General/modulefiles:/usr/local/modulefiles
PATH:
/shared/sudlab1/General/apps/conda/conda-install/envs/cgat-f/bin:/shared/sudlab1/General/apps/conda/conda-install/bin:/home/mb1ims/sh:/home/mb1ims/bin:/home/mb1ims/devel/cgat:/home/mb1ims/environment/bin:/usr/local/sge/live/bin/lx-amd64:/usr/local/scripts/:/usr/lib64/qt-3.3/bin:/scratch/2344323.1.interactive.q:/usr/local/bin:/bin:/usr/bin
REQUESTS_CA_BUNDLE: <not set>
SGE_BINARY_PATH: /usr/local/sge/live/bin/lx-amd64
SGE_CWD_PATH: /home/mb1ims
SGE_O_PATH:
/home/mb1ims/sh:/home/mb1ims/bin:/home/mb1ims/devel/cgat:/home/mb1ims/environment/bin:/usr/local/sge/live/bin/lx-amd64:/usr/local/scripts/:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/mb1ims/bin
SGE_STDERR_PATH: /dev/null
SGE_STDIN_PATH: /dev/null
SGE_STDOUT_PATH: /dev/null
SSL_CERT_FILE: <not set>


WARNING: could not import _license.show_info
# try:
# $ conda install -n root _license

Any suggests appreciated?

Yours,

Ian