[conda] conda installed inappropriate python version package

[Chenyang Yan]

I find a wired issue suddenly, when I install pytorch, conda selects Python
3 version pytorch, not Python 2:

(base) [***@21801 5.0.0]# conda install pytorch
... ...

The following NEW packages will be INSTALLED:

cudatoolkit: 9.0-h13b8566_0
cudnn: 7.1.2-cuda9.0_0
nccl: 1.3.5-cuda9.0_0
ninja: 1.8.2-py36h6bb024c_1
numpy-base: 1.14.5-py36hdbf6ddf_0
pytorch: 0.4.0-py36hdf912b8_0

The following packages will be UPDATED:

certifi: 2018.1.18-py27_intel_0 <unknown> [intel] -->
2018.4.16-py36_0
mkl_random: 1.0.1-np114py27_intel_0 <unknown> [intel] -->
1.0.1-py36h629b387_0
numpy: 1.14.3-py27_intel_0 <unknown> [intel] -->
1.14.5-py36hcd700cb_0
pip: 10.0.1-py27_0 <unknown> --> 10.0.1-py36_0
pycparser: 2.18-py27_intel_0 <unknown> [intel] -->
2.18-py36hf9f622e_1
python: 2.7.14-50078513fcfa8 <unknown> [debug] -->
3.6.5-hc3d631a_2
readline: 7.0-hac23ff0_3 <unknown> --> 7.0-ha6073c6_4
setuptools: 39.1.0-py27_0 <unknown> --> 39.2.0-py36_0
sqlite: 3.23.1-intel_0 <unknown> [intel] -->
3.24.0-h84994c4_0
tk: 8.6.4-intel_26 <unknown> [intel] -->
8.6.7-hc745277_3
wheel: 0.31.0-py27_intel_0 <unknown> [intel] -->
0.31.1-py36_0
xz: 5.2.3-intel_0 <unknown> [intel] -->
5.2.4-h14c3975_4

... ...


But conda's Python version is 2.7:
(base) [***@21801 5.0.0]# /root/222/bin/conda info

active environment : base
active env location : /root/222
shell level : 1
user config file : /root/.condarc
populated config files :
conda version : 4.5.4
conda-build version : 3.9.2
python version : 2.7.14.final.0
base environment : /root/222 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/linux-64
https://repo.anaconda.com/pkgs/pro/noarch
package cache : /root/222/pkgs
/root/.conda/pkgs
envs directories : /root/222/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/4.5.4 requests/2.18.4 CPython/2.7.14
Linux/3.10.0-693.el7.x86_64 centos/7 glibc/2.17
UID:GID : 0:0
netrc file : None
offline mode : False


I use CentOS 7.4.1708 Docker image as OS, relevant info are below following:

* Docker :17.09.0-ce
* CentOS image: https://hub.docker.com/_/centos/



What happened ? Thanks in advance .

['Chris Barker' via conda - Public]

Does pytorch have a 2.7 version? If not, then it may be installing all that
other stuff to meet its dependencies.

try:

conda info pytorch

-CHB

Post by Chenyang Yan
I find a wired issue suddenly, when I install pytorch, conda selects
... ...
cudatoolkit: 9.0-h13b8566_0
cudnn: 7.1.2-cuda9.0_0
nccl: 1.3.5-cuda9.0_0
ninja: 1.8.2-py36h6bb024c_1
numpy-base: 1.14.5-py36hdbf6ddf_0
pytorch: 0.4.0-py36hdf912b8_0
certifi: 2018.1.18-py27_intel_0 <unknown> [intel] -->
2018.4.16-py36_0
mkl_random: 1.0.1-np114py27_intel_0 <unknown> [intel] -->
1.0.1-py36h629b387_0
numpy: 1.14.3-py27_intel_0 <unknown> [intel] -->
1.14.5-py36hcd700cb_0
pip: 10.0.1-py27_0 <unknown> --> 10.0.1-py36_0
pycparser: 2.18-py27_intel_0 <unknown> [intel] -->
2.18-py36hf9f622e_1
python: 2.7.14-50078513fcfa8 <unknown> [debug] -->
3.6.5-hc3d631a_2
readline: 7.0-hac23ff0_3 <unknown> --> 7.0-ha6073c6_4
setuptools: 39.1.0-py27_0 <unknown> --> 39.2.0-py36_0
sqlite: 3.23.1-intel_0 <unknown> [intel] -->
3.24.0-h84994c4_0
tk: 8.6.4-intel_26 <unknown> [intel] -->
8.6.7-hc745277_3
wheel: 0.31.0-py27_intel_0 <unknown> [intel] -->
0.31.1-py36_0
xz: 5.2.3-intel_0 <unknown> [intel] -->
5.2.4-h14c3975_4
... ...
active environment : base
active env location : /root/222
shell level : 1
user config file : /root/.condarc
conda version : 4.5.4
conda-build version : 3.9.2
python version : 2.7.14.final.0
base environment : /root/222 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/pro/linux-64
https://repo.anaconda.com/pkgs/pro/noarch
package cache : /root/222/pkgs
/root/.conda/pkgs
envs directories : /root/222/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/4.5.4 requests/2.18.4 CPython/2.7.14
Linux/3.10.0-693.el7.x86_64 centos/7 glibc/2.17
UID:GID : 0:0
netrc file : None
offline mode : False
* Docker :17.09.0-ce
* CentOS image: https://hub.docker.com/_/centos/
What happened ? Thanks in advance .
--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/
continuum.io/d/msgid/conda/f554e63c-44e0-403d-bf62-
f85f85131916%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/conda/f554e63c-44e0-403d-bf62-f85f85131916%40continuum.io?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R [phone] voice
7600 Sand Point Way NE [phone] fax
Seattle, WA 98115 [phone] main reception

***@noaa.gov

[Chenyang Yan]

Post by 'Chris Barker' via conda - Public
Does pytorch have a 2.7 version? If not, then it may be installing all that
other stuff to meet its dependencies.

Yes, pytorch has 2.7 version, we can see it in https://repo.continuum.io/pkgs/main/linux-64/
(CTRL + F to search pytorch keyword, find 2.7 version)

Post by 'Chris Barker' via conda - Public
conda info pytorch

conda info can also get 2.7 pytorch related info, and conda search also do it.

(base) [***@21801 5.0.0]# conda search pytorch | grep py27
pytorch 0.1.12 py27_0 pkgs/free
... ...
pytorch 0.3.0 py27cuda8.0cudnn7.0_0 pkgs/main
pytorch 0.3.1 py27had95abb_0 pkgs/main
pytorch 0.4.0 py27hdf912b8_0 pkgs/main

(base) [***@21801 5.0.0]# conda info pytorch
... ...

pytorch 0.4.0 py27hdf912b8_0
----------------------------
file name : pytorch-0.4.0-py27hdf912b8_0.tar.bz2
name : pytorch
version : 0.4.0
build string: py27hdf912b8_0
build number: 0
channel : https://repo.anaconda.com/pkgs/main/linux-64
size : 224.5 MB
arch : None
constrains : ()
license : BSD 3-Clause
license_family: BSD
md5 : e7ec81123907db5024f05ee82d5146ef
platform : None
subdir : linux-64
timestamp : 1525916432634
url : https://repo.anaconda.com/pkgs/main/linux-64/pytorch-0.4.0-py27hdf912b8_0.tar.bz2
dependencies:
cffi
cudatoolkit 9.0.*
cudnn 7.*
libgcc-ng >=5.4.0
libstdcxx-ng >=5.4.0
mkl >=2018.0.2
nccl
ninja
numpy >=1.9.3,<2.0a0
python >=2.7,<2.8.0a0

... ...

[Chenyang Yan]

Finally, I find a workaround, but it is outrageous or inconceivable.

[***@21801 /]# conda --version
conda 4.5.4
[***@21801 /]# conda install pytorch
... ...
The following packages will be downloaded:

package | build
---------------------------|-----------------
pycparser-2.18 | py36hf9f622e_1 169 KB
ca-certificates-2018.03.07 | 0 124 KB
numpy-1.14.5 | py36hcd700cb_3 94 KB
cudnn-7.1.2 | cuda9.0_0 367.8 MB
xz-5.2.4 | h14c3975_4 366 KB
cffi-1.11.5 | py36h9745a5d_0 212 KB
... ...

Add debug package to solve it:
[***@21801 /]# conda install debug pytorch
... ...
The following packages will be downloaded:

package | build
---------------------------|-----------------
numpy-1.14.5 | py27hcd700cb_3 94 KB
pycparser-2.18 | py27hefa08c5_1 167 KB
numpy-base-1.14.5 | py27hdbf6ddf_3 4.1 MB
ca-certificates-2018.03.07 | 0 124 KB
mkl_random-1.0.1 | py27h629b387_0 361 KB
ninja-1.8.2 | py27h6bb024c_1 1.3 MB
... ...

Be curious for this
package(https://repo.continuum.io/pkgs/free/linux-64/debug-1.0-0.tar.bz2),
I extract this file to look up.

The similar situation is that conda-build requires conda-env, if you
use --offline option to install conda-build separetely, and then
conda-build always run fail unless installing conda-env package.

[Paul Madden]

I tried to reproduce your issue by installing a fresh Python 2.7 Miniconda
(https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh),
followed by a conda update --all --yes. And then:

% conda install pytorch
Solving environment: done

## Package Plan ##

environment location: /lscratch/pmadden/mc2

added / updated specs:
- pytorch


The following packages will be downloaded:

package | build
---------------------------|-----------------
nccl-1.3.5 | cuda9.0_0 8.6 MB
pytorch-0.4.0 | py27hdf912b8_0 224.5 MB
mkl_random-1.0.1 | py27h629b387_0 361 KB
mkl-2018.0.3 | 1 198.7 MB
numpy-1.14.5 | py27hcd700cb_3 94 KB
cudatoolkit-9.0 | h13b8566_0 340.4 MB
intel-openmp-2018.0.3 | 0 705 KB
cudnn-7.1.2 | cuda9.0_0 367.8 MB
ninja-1.8.2 | py27h6bb024c_1 1.3 MB
mkl_fft-1.0.1 | py27h3010b51_0 137 KB
blas-1.0 | mkl 6 KB
libgfortran-ng-7.2.0 | hdf63c60_3 1.2 MB
numpy-base-1.14.5 | py27hdbf6ddf_3 4.1 MB
------------------------------------------------------------
Total: 1.12 GB

The following NEW packages will be INSTALLED:

blas: 1.0-mkl
cudatoolkit: 9.0-h13b8566_0
cudnn: 7.1.2-cuda9.0_0
intel-openmp: 2018.0.3-0
libgfortran-ng: 7.2.0-hdf63c60_3
mkl: 2018.0.3-1
mkl_fft: 1.0.1-py27h3010b51_0
mkl_random: 1.0.1-py27h629b387_0
nccl: 1.3.5-cuda9.0_0
ninja: 1.8.2-py27h6bb024c_1
numpy: 1.14.5-py27hcd700cb_3
numpy-base: 1.14.5-py27hdbf6ddf_3
pytorch: 0.4.0-py27hdf912b8_0

The problem you report does not seem to be happening here, so I would guess
that there are other packages already installed in your environment that
are leading to the strange environment solution conda is suggesting. It
could be that there are upstream packages with incorrectly defined
dependencies -- that happens sometimes -- or just that the combination of
ALL the packages in your base environment is such that conda finds a
solution that involves upgrading to Python 3.

In my experience, there is always a good explanation for behavior like
this, but it can involving investigating the recipes of many packages to
understand what is happening. Also in my experience, the more packages in
the environment, the more likely it is that conda will find an unexpected
(and even unwanted) solution. What works best for me is to keep my base
environment minimal, and create small environments specific to certain
tasks. In fact, I never install anything into my base environment except
the conda-build package, which I use often. Otherwise, for task-specific
work, I create separate environments with exactly the packages I need for
that task.

You could try, for example, to create an isolated environment for working
with pytorch, e.g.

% conda create -n pytorch pytorch [ any other packages you require when
working with pytorch ]

and see if your issue disappears.

[Chenyang Yan]

圚 2018幎7月5日星期四 UTC+8䞊午5:05:56Paul Madden写道

Post by Paul Madden
The problem you report does not seem to be happening here, so I would
guess that there are other packages already installed in your environment
that are leading to the strange environment solution conda is suggesting.
It could be that there are upstream packages with incorrectly defined
dependencies -- that happens sometimes -- or just that the combination of
ALL the packages in your base environment is such that conda finds a
solution that involves upgrading to Python 3.

Firstly, thanks very much for your time to investigate my issue ! I
agree that some package has incorrectly defined dependencies.

Post by Paul Madden
In my experience, there is always a good explanation for behavior like
this, but it can involving investigating the recipes of many packages to
understand what is happening. Also in my experience, the more packages in
the environment, the more likely it is that conda will find an unexpected
(and even unwanted) solution. What works best for me is to keep my base
environment minimal, and create small environments specific to certain
tasks. In fact, I never install anything into my base environment except
the conda-build package, which I use often. Otherwise, for task-specific
work, I create separate environments with exactly the packages I need for
that task.
You could try, for example, to create an isolated environment for working
with pytorch, e.g.
% conda create -n pytorch pytorch [ any other packages you require when
working with pytorch ]

Good idea ! one environment, do one thing. I will hear your view .
Thanks again !

Post by Paul Madden
and see if your issue disappears.