[conda] using conda cli as an api (--json, etc)

[Chris Withers]

Hi All,

I see the conda cli now has --json and I remember being told in the past
that conda would never be available as a library, should you want to
introspect stuff from your environment, and that you should just shell
out to conda in a subprocess...

...but now it appears that conda is just a bash function, and it seems
obnoxiously hard to shell out to conda in a subprocess.

What am I missing?

Chris

[Chris Withers]

Post by Chris Withers
but now it appears that conda is just a bash function

"Just a bash function" is incorrect.  Conda is now a shell function
first.  The conda executable is there and in the same place that it
always has been.

Post by Chris Withers
it seems obnoxiously hard to shell out to conda in a subprocess.

Is the issue that you're not using the full path to the conda
executable, or that PATH resolution doesn't work because you're not
activating the base environment?

Let me explain by way of a shell session:

(picky-conda) tweedledee:PyCharm2018.1 chris$ conda env export --json
name: picky-conda
channels:
- defaults
- conda-forge
dependencies:
...
prefix: /Users/chris/anaconda2/envs/picky-conda


Right, great, so I want the output of that to use progamatically

(picky-conda) $ python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:14:23)
...

Post by Chris Withers

from subprocess import check_output
print(check_output(['conda', 'env', 'export']))

Traceback (most recent call last):
...
FileNotFoundError: [Errno 2] No such file or directory: 'conda': 'conda'

Um, wut?

(picky-conda) $ type -a conda
conda is a function
...
install | update | uninstall | remove)
$_CONDA_EXE "$cmd" "$@" && _conda_reactivate
...


Okay, well, that's not exactly friendly, but sure:

(picky-conda) $ echo $_CONDA_EXE
/Users/chris/anaconda2/bin/conda
(picky-conda) $ python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:14:23)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Post by Chris Withers

import os
os.environ.get('_CONDA_EXE')
os.environ.get('_CONDA_EXE') is None

True

Um, wut, wut?

Post by Chris Withers

print(check_output(['/Users/chris/anaconda2/bin/conda', 'env',

'export']).decode('utf-8'))
name: picky-conda
channels:
- defaults
- conda-forge
dependencies:
...at last...
prefix: /Users/chris/anaconda2/envs/picky-conda

Even if it's not really of use to me, because I don't have a good way to
really find out where conda is.

Post by Chris Withers
and I remember being told in the past that conda would never be

available as a library
Never say never.  See [Supported ways to use conda as a python
library](https://github.com/conda/conda/issues/7059).

I don't understand: pip is now breaking code that uses pip as a library
so conda is adding support for use of conda as a library?

Chris

[Chris Withers]

I never considered a `bin/conda` symlink to be part of the definition of
a conda environment.

I'm not sure what you mean? I just want to be able to run 'conda env
export' in a subprocess, since I was lead to believe that was the
approved and supported api for obtaining that information.

people did, and I guess that was a blind spot of mine.  FWIW, I've
always considered the existence of a `conda-meta/history` file to be the
sole indicator of a conda environment.

Okay, but not sure how that relates to what I'm trying to do?

$_CONDA_EXE isn't there because it's not exported.  See
https://github.com/conda/conda/issues/7126.  That'll be fixed in 4.6.
It'll be exported as $CONDA_EXE for sure, and we might do $_CONDA_EXE to
maintain compatibility, although since $_CONDA_EXE was never exported,
I'm not sure that's necessary.

So os.environ.get('CONDA_EXE') is supported and going to work for the
forseable future?

cheers,

Chris

[Shaun Walbridge]

If you want to go one step further and just rely on the environment state
and not environment variables, try this:

import subprocess
conda_path = subprocess.check_output(('which',
'conda')).strip().decode("utf-8")

Post by Chris Withers

I never considered a `bin/conda` symlink to be part of the definition of
a conda environment.

I'm not sure what you mean? I just want to be able to run 'conda env
export' in a subprocess, since I was lead to believe that was the approved
and supported api for obtaining that information.
people did, and I guess that was a blind spot of mine. FWIW, I've always

considered the existence of a `conda-meta/history` file to be the sole
indicator of a conda environment.

Okay, but not sure how that relates to what I'm trying to do?
$_CONDA_EXE isn't there because it's not exported. See

https://github.com/conda/conda/issues/7126. That'll be fixed in 4.6.
It'll be exported as $CONDA_EXE for sure, and we might do $_CONDA_EXE to
maintain compatibility, although since $_CONDA_EXE was never exported, I'm
not sure that's necessary.

So os.environ.get('CONDA_EXE') is supported and going to work for the
forseable future?
cheers,
Chris
--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/co
ntinuum.io/d/msgid/conda/195f1643-1ad2-f89d-de28-e83d3952830
e%40withers.org.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[Chris Withers]

Post by Shaun Walbridge
If you want to go one step further and just rely on the environment
import subprocess
conda_path = subprocess.check_output(('which',
'conda')).strip().decode("utf-8")

I'm afraid you're mistaken, assuming you have a new version of conda and
have cleaned up your shell profiles as directed by 'conda activate
your_env':

tweedledee:picky-conda chris$ conda activate picky-conda
(picky-conda) tweedledee:picky-conda chris$ python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:14:23)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Post by Shaun Walbridge

Post by Shaun Walbridge
import subprocess
conda_path = subprocess.check_output(('which',

'conda')).strip().decode("utf-8")
Traceback (most recent call last):
...
subprocess.CalledProcessError: Command '('which', 'conda')' returned
non-zero exit status 1.

If that were to work, I wouldn't need to do anything other than the
check_output(['conda', 'env', 'export']) I actually want to do ;-)

Chris

[Shaun Walbridge]

Strange, it does work here:

$ conda activate pyf
(pyf) ***@mx ~
$ which conda
/Users/scw/miniconda3/envs/pyf/bin/conda
(pyf) ***@mx ~
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:14:23)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Post by Chris Withers

Post by Chris Withers

Post by Shaun Walbridge
import subprocess
conda_path = subprocess.check_output(('which',

'conda')).strip().decode("utf-8")

Post by Chris Withers

Post by Chris Withers

Post by Shaun Walbridge
conda_path

'/Users/scw/miniconda3/envs/pyf/bin/conda'

In this case, using the .bash_profile of
. /Users/scw/miniconda3/etc/profile.d/conda.sh

But miniconda also uses this form which should work just fine as well:
export PATH="/path/to/conda/bin:$PATH"

I also tried it with the (old) activate script (source activate && activate
env) and it worked fine there as well, without anything being in my
.bash_profile.

Post by Chris Withers

Post by Chris Withers
have cleaned up your shell profiles as directed by 'conda activate

your_env'
Yeah, to be equivalent, we really should have added `&& conda activate
base` to those instructions. All of that is why I'm eager to get
https://github.com/conda/conda/pull/6518 out with conda 4.6 and finally
have all of this shell activation work complete. At least a full
completion of the first phase, before we move on to
https://github.com/conda/conda/issues/6820.

Post by Chris Withers

Post by Shaun Walbridge
If you want to go one step further and just rely on the environment
import subprocess
conda_path = subprocess.check_output(('which',
'conda')).strip().decode("utf-8")

I'm afraid you're mistaken, assuming you have a new version of conda and
have cleaned up your shell profiles as directed by 'conda activate
tweedledee:picky-conda chris$ conda activate picky-conda
(picky-conda) tweedledee:picky-conda chris$ python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:14:23)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Post by Shaun Walbridge

Post by Shaun Walbridge
import subprocess
conda_path = subprocess.check_output(('which',

'conda')).strip().decode("utf-8")
...
subprocess.CalledProcessError: Command '('which', 'conda')' returned
non-zero exit status 1.
If that were to work, I wouldn't need to do anything other than the
check_output(['conda', 'env', 'export']) I actually want to do ;-)
Chris

[Chris Withers]

Post by Shaun Walbridge
In this case, using the .bash_profile of
. /Users/scw/miniconda3/etc/profile.d/conda.sh
export PATH="/path/to/conda/bin:$PATH"

What version of conda are you using? I'm on 4.5.0 and when I tried
'conda activate myenv' it told me to remove the above lines...

Chris

[Shaun Walbridge]

I'm also on 4.5.0 as well. I think as Kale described, the instructions
aren't quite perfect ye, but till 4.6 is released the other approaches I
mentioned should work.

Post by Shaun Walbridge
In this case, using the .bash_profile of
. /Users/scw/miniconda3/etc/profile.d/conda.sh
export PATH="/path/to/conda/bin:$PATH"

What version of conda are you using? I'm on 4.5.0 and when I tried 'conda
activate myenv' it told me to remove the above lines...
Chris

[Chris Withers]

Post by Chris Withers

Post by Chris Withers
have cleaned up your shell profiles as directed by 'conda activate

your_env'
Yeah, to be equivalent, we really should have added `&& conda activate
base` to those instructions.

The idea of having more than one environment activated at the same time
seems like a really bad idea. What's the driver for every supporting that?

Chris