[conda] pyarrow installed from conda-forge while it seems to exist in the default channel

[Fabien Tarrade]

Hi there,

I am creating an conda env using a yml file that look like that:

channels:
- defaults
- pytorch
- conda-forge
...

- pyarrow=0.11.0
- arrow-cpp=0.11.0
..

I got the following packages installed:

arrow-cpp                 0.11.0               py36h5c3f529_0
boost-cpp                 1.67.0               h14c3975_4
libboost                    1.67.0                h46d08c1_4
pyarrow                    0.11.0                py36hfc679d8_0 conda-forge

What trigger to have pyarrow installed from the conda-forge channel
while it exist in the default channel (for the same version) ? It look
that mixing the channel for pyarrow and arrow-cpp create some issue.

https://anaconda.org/conda-forge/arrow-cpp
https://anaconda.org/anaconda/arrow-cpp

arrow-cpp 0.11.0 py36h5c3f529_0
-------------------------------
file name   : arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
name        : arrow-cpp
version     : 0.11.0
build string: py36h5c3f529_0
build number: 0
channel     : https://repo.anaconda.com/pkgs/main/linux-64
size        : 6.7 MB
arch        : None
constrains  : ()
license     : Apache 2.0
md5         : de8456a017a8e9c01d4c2b2e1d2b8b56
platform    : None
subdir      : linux-64
timestamp   : 1539967743950
url         :
https://repo.anaconda.com/pkgs/main/linux-64/arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
dependencies:
    glog >=0.3.5,<0.3.6.0a0
    libboost >=1.67.0,<1.67.1.0a0
    libgcc-ng >=7.3.0
    libstdcxx-ng >=7.3.0
    lz4-c >=1.8.1.2,<1.9.0a0
    numpy >=1.11.3,<2.0a0
    python >=3.6,<3.7.0a0
    snappy >=1.1.7,<2.0a0
    thrift-cpp >=0.11.0,<0.12.0a0
    zlib >=1.2.11,<1.3.0a0
    zstd >=1.3.3,<1.3.4.0a0

https://anaconda.org/conda-forge/pyarrow
https://anaconda.org/anaconda/pyarrow

pyarrow 0.11.0 py27hfc679d8_0
-----------------------------
file name   : pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
name        : pyarrow
version     : 0.11.0
build string: py27hfc679d8_0
build number: 0
channel     : https://conda.anaconda.org/conda-forge/linux-64
size        : 1.9 MB
arch        : x86_64
constrains  : ()
license     : Apache 2.0
md5         : d88826f8a8d33d04574e5916ffa441e1
platform    : linux
subdir      : linux-64
timestamp   : 1539093791645
url         :
https://conda.anaconda.org/conda-forge/linux-64/pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
dependencies:
    arrow-cpp 0.11.0.*
    futures
    libgcc-ng >=4.9
    libstdcxx-ng >=4.9
    numpy >=1.14,<2.0a0
    pandas
    python >=2.7,<2.8.0a0
    setuptools
    six

The issue is describe here:
https://github.com/apache/arrow/issues/2213

I can fix the issue it by creating my env and then updating the package
with the right channel and but is not super nice and pratical.

conda install arrow-cpp -c conda-forge

I am using Centos centos:6.9
I am always using the latest version of conda and python 3.6.6

Thanks
Cheers
Fabien
--
Dr. Fabien Tarrade

Senior Data Scientist at AXA
<https://www.facebook.com/fabien.tarrade.eu>

I am a senior Data Scientist at AXA with the mission of helping AXA
becoming a data driven organisation by using advanced analytics and Big
Data.
I have over 10 years of experience in management of large projects,
processing, modelling and statistical treatment of large volume of
experimental data
up to 10 petabytes as well as the development and maintenance of
advanced and complex computer programs.

Zurich, Switzerland

Email : ***@fabien-tarrade.eu <mailto:***@fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>

LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter
<https://twitter.com/fabtar> Google
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook
<https://www.facebook.com/fabien.tarrade.eu> Google
<skype:fabtarhiggs?call> Xing <https://www.xing.com/profile/Fabien_Tarrade>

[Mário Costa]

please use conda config --remove channels defaults, consider also checking
the order of the channels using --prepend and --append, as they influence
priority/version matching of package dependency algorithim.

Yeah, that's what he's trying to do. He's confused on why conda is giving
him a mixed solution.

Hi Fabien,
You cannot mix packages from Anaconda Distribution defaults and those
from conda-forge in the same environment at present as we are not binary
compatible.
Please use one or the other in any given environment.

Post by Fabien Tarrade
Hi there,
- defaults
- pytorch
- conda-forge
...
- pyarrow=0.11.0
- arrow-cpp=0.11.0
..
arrow-cpp 0.11.0 py36h5c3f529_0
boost-cpp 1.67.0 h14c3975_4
libboost 1.67.0 h46d08c1_4
pyarrow 0.11.0 py36hfc679d8_0
conda-forge
What trigger to have pyarrow installed from the conda-forge channel
while it exist in the default channel (for the same version) ? It look that
mixing the channel for pyarrow and arrow-cpp create some issue.
https://anaconda.org/conda-forge/arrow-cpp
https://anaconda.org/anaconda/arrow-cpp
arrow-cpp 0.11.0 py36h5c3f529_0
-------------------------------
file name : arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
name : arrow-cpp
version : 0.11.0
build string: py36h5c3f529_0
build number: 0
channel : https://repo.anaconda.com/pkgs/main/linux-64
size : 6.7 MB
arch : None
constrains : ()
license : Apache 2.0
md5 : de8456a017a8e9c01d4c2b2e1d2b8b56
platform : None
subdir : linux-64
timestamp : 1539967743950
https://repo.anaconda.com/pkgs/main/linux-64/arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
glog >=0.3.5,<0.3.6.0a0
libboost >=1.67.0,<1.67.1.0a0
libgcc-ng >=7.3.0
libstdcxx-ng >=7.3.0
lz4-c >=1.8.1.2,<1.9.0a0
numpy >=1.11.3,<2.0a0
python >=3.6,<3.7.0a0
snappy >=1.1.7,<2.0a0
thrift-cpp >=0.11.0,<0.12.0a0
zlib >=1.2.11,<1.3.0a0
zstd >=1.3.3,<1.3.4.0a0
https://anaconda.org/conda-forge/pyarrow
https://anaconda.org/anaconda/pyarrow
pyarrow 0.11.0 py27hfc679d8_0
-----------------------------
file name : pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
name : pyarrow
version : 0.11.0
build string: py27hfc679d8_0
build number: 0
channel : https://conda.anaconda.org/conda-forge/linux-64
size : 1.9 MB
arch : x86_64
constrains : ()
license : Apache 2.0
md5 : d88826f8a8d33d04574e5916ffa441e1
platform : linux
subdir : linux-64
timestamp : 1539093791645
https://conda.anaconda.org/conda-forge/linux-64/pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
arrow-cpp 0.11.0.*
futures
libgcc-ng >=4.9
libstdcxx-ng >=4.9
numpy >=1.14,<2.0a0
pandas
python >=2.7,<2.8.0a0
setuptools
six
https://github.com/apache/arrow/issues/2213
I can fix the issue it by creating my env and then updating the package
with the right channel and but is not super nice and pratical.
conda install arrow-cpp -c conda-forge
I am using Centos centos:6.9
I am always using the latest version of conda and python 3.6.6
Thanks
Cheers
Fabien
--
Dr. Fabien Tarrade
Senior Data Scientist at AXA
<https://www.facebook.com/fabien.tarrade.eu>
I am a senior Data Scientist at AXA with the mission of helping AXA
becoming a data driven organisation by using advanced analytics and Big
Data.
I have over 10 years of experience in management of large projects,
processing, modelling and statistical treatment of large volume of
experimental data
up to 10 petabytes as well as the development and maintenance of
advanced and complex computer programs.
Zurich, Switzerland
[image: Phone] : www.fabien-tarrade.eu
Twitter] <https://twitter.com/fabtar> [image: Google]
<https://plus.google.com/+FabienTarradeProfile/posts> [image: Facebook]
Xing] <https://www.xing.com/profile/Fabien_Tarrade>
--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at
https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/7d255e18-f303-4091-8fbd-48947c312d6a%40gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/7d255e18-f303-4091-8fbd-48947c312d6a%40gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit
https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/
.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/CAF5kcVXsQ7We%2B2PtU4_AVOdjrRaaWX63zD-K2T5A4rCfjrDXmg%40mail.gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/CAF5kcVXsQ7We%2B2PtU4_AVOdjrRaaWX63zD-K2T5A4rCfjrDXmg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/a/continuum.io/d/optout
.

--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/CAOQN5Ogx7JcV3o_tjPe4A-QyqK4hQMr0YUedDDrkrvj0fkX-rg%40mail.gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/CAOQN5Ogx7JcV3o_tjPe4A-QyqK4hQMr0YUedDDrkrvj0fkX-rg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[Mário Costa]

consider reading this
https://conda.io/docs/user-guide/tasks/manage-channels.html.

Post by Mário Costa
please use conda config --remove channels defaults, consider also checking
the order of the channels using --prepend and --append, as they influence
priority/version matching of package dependency algorithim.

Yeah, that's what he's trying to do. He's confused on why conda is
giving him a mixed solution.

Hi Fabien,
You cannot mix packages from Anaconda Distribution defaults and those
from conda-forge in the same environment at present as we are not binary
compatible.
Please use one or the other in any given environment.
On Sat, Oct 27, 2018 at 10:23 PM Fabien Tarrade <

Post by Fabien Tarrade
Hi there,
- defaults
- pytorch
- conda-forge
...
- pyarrow=0.11.0
- arrow-cpp=0.11.0
..
arrow-cpp 0.11.0 py36h5c3f529_0
boost-cpp 1.67.0 h14c3975_4
libboost 1.67.0 h46d08c1_4
pyarrow 0.11.0 py36hfc679d8_0
conda-forge
What trigger to have pyarrow installed from the conda-forge channel
while it exist in the default channel (for the same version) ? It look that
mixing the channel for pyarrow and arrow-cpp create some issue.
https://anaconda.org/conda-forge/arrow-cpp
https://anaconda.org/anaconda/arrow-cpp
arrow-cpp 0.11.0 py36h5c3f529_0
-------------------------------
file name : arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
name : arrow-cpp
version : 0.11.0
build string: py36h5c3f529_0
build number: 0
channel : https://repo.anaconda.com/pkgs/main/linux-64
size : 6.7 MB
arch : None
constrains : ()
license : Apache 2.0
md5 : de8456a017a8e9c01d4c2b2e1d2b8b56
platform : None
subdir : linux-64
timestamp : 1539967743950
https://repo.anaconda.com/pkgs/main/linux-64/arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
glog >=0.3.5,<0.3.6.0a0
libboost >=1.67.0,<1.67.1.0a0
libgcc-ng >=7.3.0
libstdcxx-ng >=7.3.0
lz4-c >=1.8.1.2,<1.9.0a0
numpy >=1.11.3,<2.0a0
python >=3.6,<3.7.0a0
snappy >=1.1.7,<2.0a0
thrift-cpp >=0.11.0,<0.12.0a0
zlib >=1.2.11,<1.3.0a0
zstd >=1.3.3,<1.3.4.0a0
https://anaconda.org/conda-forge/pyarrow
https://anaconda.org/anaconda/pyarrow
pyarrow 0.11.0 py27hfc679d8_0
-----------------------------
file name : pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
name : pyarrow
version : 0.11.0
build string: py27hfc679d8_0
build number: 0
channel : https://conda.anaconda.org/conda-forge/linux-64
size : 1.9 MB
arch : x86_64
constrains : ()
license : Apache 2.0
md5 : d88826f8a8d33d04574e5916ffa441e1
platform : linux
subdir : linux-64
timestamp : 1539093791645
https://conda.anaconda.org/conda-forge/linux-64/pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
arrow-cpp 0.11.0.*
futures
libgcc-ng >=4.9
libstdcxx-ng >=4.9
numpy >=1.14,<2.0a0
pandas
python >=2.7,<2.8.0a0
setuptools
six
https://github.com/apache/arrow/issues/2213
I can fix the issue it by creating my env and then updating the package
with the right channel and but is not super nice and pratical.
conda install arrow-cpp -c conda-forge
I am using Centos centos:6.9
I am always using the latest version of conda and python 3.6.6
Thanks
Cheers
Fabien
--
Dr. Fabien Tarrade
Senior Data Scientist at AXA
<https://www.facebook.com/fabien.tarrade.eu>
I am a senior Data Scientist at AXA with the mission of helping AXA
becoming a data driven organisation by using advanced analytics and Big
Data.
I have over 10 years of experience in management of large projects,
processing, modelling and statistical treatment of large volume of
experimental data
up to 10 petabytes as well as the development and maintenance of
advanced and complex computer programs.
Zurich, Switzerland
[image: Phone] : www.fabien-tarrade.eu
Twitter] <https://twitter.com/fabtar> [image: Google]
<https://plus.google.com/+FabienTarradeProfile/posts> [image: Facebook]
Xing] <https://www.xing.com/profile/Fabien_Tarrade>
--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at
https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/7d255e18-f303-4091-8fbd-48947c312d6a%40gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/7d255e18-f303-4091-8fbd-48947c312d6a%40gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit
https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at
https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/CAF5kcVXsQ7We%2B2PtU4_AVOdjrRaaWX63zD-K2T5A4rCfjrDXmg%40mail.gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/CAF5kcVXsQ7We%2B2PtU4_AVOdjrRaaWX63zD-K2T5A4rCfjrDXmg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit
https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/
.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/CAOQN5Ogx7JcV3o_tjPe4A-QyqK4hQMr0YUedDDrkrvj0fkX-rg%40mail.gmail.com
<https://groups.google.com/a/continuum.io/d/msgid/conda/CAOQN5Ogx7JcV3o_tjPe4A-QyqK4hQMr0YUedDDrkrvj0fkX-rg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/a/continuum.io/d/optout
.

[Fabien Tarrade]

Hi Mário, Michael and Ray,

Thanks a lot for your input.

From the doc, it is possible to mixed channel. I am doing that since
few year for some env with quite a lot of packages. I am always trying
to get packages from one main channel but I also need to install some
packages with pip and from other channel. Any new version of conda
improved such issue. Great.

I saw other discussion and the best solution is to drop the default
channel as first choice and to use conda-forge. By doing so, I lot of
compatibility issue are now gone.

Thanks
Cheers
Fabien

Post by Mário Costa
consider reading this
https://conda.io/docs/user-guide/tasks/manage-channels.html.
On Sun, Oct 28, 2018 at 12:14 AM Mário Costa
please use conda config --remove channels defaults, consider also
checking the order of the channels using --prepend and --append,
as they influence priority/version matching of package dependency
algorithim.
On Sat, Oct 27, 2018 at 10:35 PM Michael Sarahan
Yeah, that's what he's trying to do.  He's confused on why
conda is giving him a mixed solution.
On Sat, Oct 27, 2018 at 5:33 PM Ray Donnelly
Hi Fabien,
You cannot mix packages from Anaconda Distribution
defaults and those from conda-forge in the same
environment at present as we are not binary compatible.
Please use one or the other in any given environment.
On Sat, Oct 27, 2018 at 10:23 PM Fabien Tarrade
Hi there,
I am creating an conda env using a yml file that look
- defaults
- pytorch
- conda-forge
...
- pyarrow=0.11.0
- arrow-cpp=0.11.0
..
arrow-cpp 0.11.0               py36h5c3f529_0
boost-cpp                 1.67.0 h14c3975_4
libboost 1.67.0                h46d08c1_4
pyarrow 0.11.0                py36hfc679d8_0 conda-forge
What trigger to have pyarrow installed from the
conda-forge channel while it exist in the default
channel (for the same version) ? It look that mixing
the channel for pyarrow and arrow-cpp create some issue.
https://anaconda.org/conda-forge/arrow-cpp
https://anaconda.org/anaconda/arrow-cpp
arrow-cpp 0.11.0 py36h5c3f529_0
-------------------------------
file name   : arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
name        : arrow-cpp
version     : 0.11.0
build string: py36h5c3f529_0
build number: 0
channel     : https://repo.anaconda.com/pkgs/main/linux-64
size        : 6.7 MB
arch        : None
constrains  : ()
license     : Apache 2.0
md5         : de8456a017a8e9c01d4c2b2e1d2b8b56
platform    : None
subdir      : linux-64
timestamp   : 1539967743950
https://repo.anaconda.com/pkgs/main/linux-64/arrow-cpp-0.11.0-py36h5c3f529_0.tar.bz2
    glog >=0.3.5,<0.3.6.0a0
    libboost >=1.67.0,<1.67.1.0a0
    libgcc-ng >=7.3.0
    libstdcxx-ng >=7.3.0
    lz4-c >=1.8.1.2,<1.9.0a0
    numpy >=1.11.3,<2.0a0
    python >=3.6,<3.7.0a0
    snappy >=1.1.7,<2.0a0
    thrift-cpp >=0.11.0,<0.12.0a0
    zlib >=1.2.11,<1.3.0a0
    zstd >=1.3.3,<1.3.4.0a0
https://anaconda.org/conda-forge/pyarrow
https://anaconda.org/anaconda/pyarrow
pyarrow 0.11.0 py27hfc679d8_0
-----------------------------
file name   : pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
name        : pyarrow
version     : 0.11.0
build string: py27hfc679d8_0
build number: 0
https://conda.anaconda.org/conda-forge/linux-64
size        : 1.9 MB
arch        : x86_64
constrains  : ()
license     : Apache 2.0
md5         : d88826f8a8d33d04574e5916ffa441e1
platform    : linux
subdir      : linux-64
timestamp   : 1539093791645
https://conda.anaconda.org/conda-forge/linux-64/pyarrow-0.11.0-py27hfc679d8_0.tar.bz2
    arrow-cpp 0.11.0.*
    futures
    libgcc-ng >=4.9
    libstdcxx-ng >=4.9
    numpy >=1.14,<2.0a0
    pandas
    python >=2.7,<2.8.0a0
    setuptools
    six
https://github.com/apache/arrow/issues/2213
I can fix the issue it by creating my env and then
updating the package with the right channel and but is
not super nice and pratical.
conda install arrow-cpp -c conda-forge
I am using Centos centos:6.9
I am always using the latest version of conda and
python 3.6.6
Thanks
Cheers
Fabien

--
Dr. Fabien Tarrade

Senior Data Scientist at AXA
<https://www.facebook.com/fabien.tarrade.eu>

I am a senior Data Scientist at AXA with the mission of helping AXA
becoming a data driven organisation by using advanced analytics and Big
Data.
I have over 10 years of experience in management of large projects,
processing, modelling and statistical treatment of large volume of
experimental data
up to 10 petabytes as well as the development and maintenance of
advanced and complex computer programs.

Zurich, Switzerland

Email : ***@fabien-tarrade.eu <mailto:***@fabien-tarrade.eu>
Phone : www.fabien-tarrade.eu <http://www.fabien-tarrade.eu>


LinkedIn <http://ch.linkedin.com/in/fabientarrade/> Twitter
<https://twitter.com/fabtar> Google
<https://plus.google.com/+FabienTarradeProfile/posts> Facebook
<https://www.facebook.com/fabien.tarrade.eu> Google
<skype:fabtarhiggs?call> Xing <https://www.xing.com/profile/Fabien_Tarrade>