[conda] Feature request: would it be possible to expose ProgressiveFetchExtract's 'ProgressBar' (or 'update_to')?

[Stephen Waterbury]

Hi conda developers,

First, let me convey a huge thanks (again) for conda-constructor! I have
used it to create an installer for my pyqt gui app, which is now
"self-updating" using conda. Using conda and conda-constructor is by *FAR*
the best way to package and deploy a large gui-based app that uses SciPy
libraries: for python-illiterate users (the majority, for my app), they
don't have to know that conda is the source of the magic; OTOH, my
python-literate users get a nice conda environment along with my app, for
free! ;)

I've given my application an "update" menu function that internally calls
'conda.cli.main' in a separate thread to run 'update' (or 'install' or
'list'), which is a HUGE benefit to my app because its constructor-built
installer is over 500MB (some huge dependencies), but my app's package is
relatively tiny (< 4MB, of which the bulk is images/icons) so it updates
relatively quickly and users don't have to uninstall and run a new
installer for every update (which would be horrible).

Currently I just have my progress bar in cyclic mode, which is okay but
could be a little prettier if I could callback to my progress bar with the
progress from conda install. Looking through the conda code, it appears
that I could do that if I could give ProgressiveFetchExtract a "proxy"
ProgressBar and/or utilize its 'update_to' callback to update my pyqt
progress bar.

I guess one way might be to give ProgressiveFetchExtract a
'progress_bar_class' kwarg so I could pass in a class to use. (Maybe too
hacky?) But of course it would somehow have to be accessible via
conda.cli.main or something ... probably not a high priority item but would
make conda really cool for self-updating gui applications!

Cheers,
Steve

[Steve Waterbury]

Great, I was unaware of that stuff, thanks!  Is it documented somewhere?
("Use the code, Luke!" is a perfectly fine response... just thought I'd
ask. ;)

But why the "If you /really/ insist ..." ?  You think my "self-updating
app" is a badidea?
Gee, I thought it was brilliant ... :D

Also: "... it might not always be written in python" ... ok, I can do
subprocess,
not really aproblem.  Is there some motivation to write conda in a language
other than python? I'm surprised, to say the least!

Steve

If you /really/ insist on import conda, then it's probably best to
restrict yourself to (1) conda.cli.python_api, (2) conda.api, and, if
you must, (3) conda.exports.  Anaconda Navigator makes extensive use
of conda, actually by subprocessing to conda (hey, it might not always
be written in python!), and using the --json flag to get
machine-friendly structured output.  Progress bar output is included
in that json output.
On Tuesday, January 30, 2018 at 2:36:18 PM UTC-6, Stephen Waterbury
Hi conda developers,
First, let me convey a huge thanks (again) for conda-constructor! 
I have used it to create an installer for my pyqt gui app, which
is now "self-updating" using conda.  Using conda and
conda-constructor is by *FAR* the best way to package and deploy a
large gui-based app that uses SciPy libraries:  for
python-illiterate users (the majority, for my app), they don't
have to know that conda is the source of the magic; OTOH, my
python-literate users get a nice conda environment along with my
app, for free!  ;)
I've given my application an "update" menu function that
internally calls 'conda.cli.main' in a separate thread to run
'update' (or 'install' or 'list'), which is a HUGE benefit to my
app because its constructor-built installer is over 500MB (some
huge dependencies), but my app's package is relatively tiny (<
4MB, of which the bulk is images/icons) so it updates relatively
quickly and users don't have to uninstall and run a new installer
for every update (which would be horrible).
Currently I just have my progress bar in cyclic mode, which is
okay but could be a little prettier if I could callback to my
progress bar with the progress from conda install.  Looking
through the conda code, it appears that I could do that if I could
give ProgressiveFetchExtract a "proxy" ProgressBar and/or utilize
its 'update_to' callback to update my pyqt progress bar.
I guess one way might be to give ProgressiveFetchExtract a
'progress_bar_class' kwarg so I could pass in a class to use. 
(Maybe too hacky?)  But of course it would somehow have to be
accessible via conda.cli.main or something ... probably not a high
priority item but would make conda really cool for self-updating
gui applications!
Cheers,
Steve
--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/e4cab951-70af-4e09-ae7d-da783891a736%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/conda/e4cab951-70af-4e09-ae7d-da783891a736%40continuum.io?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[Steve Waterbury]

Never mind about docs -- I just tried it and it's sufficiently
"self-documenting".

So I hope all you thought was a bad idea was 'import conda' -- yes, it is
unnecessary now, and I'll be ready if you re-implement conda in Lisp
and deploy it as an executable.  :)

Thanks again,
Steve

Post by Steve Waterbury
Great, I was unaware of that stuff, thanks!  Is it documented somewhere?
("Use the code, Luke!" is a perfectly fine response... just thought
I'd ask. ;)
But why the "If you /really/ insist ..." ?  You think my
"self-updating app" is a badidea?
Gee, I thought it was brilliant ... :D
Also: "... it might not always be written in python" ... ok, I can do
subprocess,
not really aproblem.  Is there some motivation to write conda in a language
other than python?  I'm surprised, to say the least!
Steve

If you /really/ insist on import conda, then it's probably best to
restrict yourself to (1) conda.cli.python_api, (2) conda.api, and, if
you must, (3) conda.exports.  Anaconda Navigator makes extensive use
of conda, actually by subprocessing to conda (hey, it might not
always be written in python!), and using the --json flag to get
machine-friendly structured output.  Progress bar output is included
in that json output.
On Tuesday, January 30, 2018 at 2:36:18 PM UTC-6, Stephen Waterbury
Hi conda developers,
First, let me convey a huge thanks (again) for
conda-constructor!  I have used it to create an installer for my
pyqt gui app, which is now "self-updating" using conda.  Using
conda and conda-constructor is by *FAR* the best way to package
and deploy a large gui-based app that uses SciPy libraries:  for
python-illiterate users (the majority, for my app), they don't
have to know that conda is the source of the magic; OTOH, my
python-literate users get a nice conda environment along with my
app, for free!  ;)
I've given my application an "update" menu function that
internally calls 'conda.cli.main' in a separate thread to run
'update' (or 'install' or 'list'), which is a HUGE benefit to my
app because its constructor-built installer is over 500MB (some
huge dependencies), but my app's package is relatively tiny (<
4MB, of which the bulk is images/icons) so it updates relatively
quickly and users don't have to uninstall and run a new installer
for every update (which would be horrible).
Currently I just have my progress bar in cyclic mode, which is
okay but could be a little prettier if I could callback to my
progress bar with the progress from conda install.  Looking
through the conda code, it appears that I could do that if I
could give ProgressiveFetchExtract a "proxy" ProgressBar and/or
utilize its 'update_to' callback to update my pyqt progress bar.
I guess one way might be to give ProgressiveFetchExtract a
'progress_bar_class' kwarg so I could pass in a class to use. 
(Maybe too hacky?)  But of course it would somehow have to be
accessible via conda.cli.main or something ... probably not a
high priority item but would make conda really cool for
self-updating gui applications!
Cheers,
Steve
--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at
https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/e4cab951-70af-4e09-ae7d-da783891a736%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/conda/e4cab951-70af-4e09-ae7d-da783891a736%40continuum.io?utm_medium=email&utm_source=footer>.
For more options, visit
https://groups.google.com/a/continuum.io/d/optout.

--
You received this message because you are subscribed to the Google
Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit
https://groups.google.com/a/continuum.io/d/msgid/conda/b346d2d6-1262-1ca8-2fdc-b13d4422af17%40pangalactic.us
<https://groups.google.com/a/continuum.io/d/msgid/conda/b346d2d6-1262-1ca8-2fdc-b13d4422af17%40pangalactic.us?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[Thomas Holder]

[...] Anaconda Navigator makes extensive use of conda, actually by subprocessing to conda (hey, it might not always be written in python!), and using the --json flag to get machine-friendly structured output. [...]

I also like to do this but found that "conda" is a bash function in conda 4.4 and trying to execute "conda" in a subprocess will fail if working in a (non-base) environment. With conda 4.3, there was actually a "conda" executable in the environment's "bin" directory.

# 1) activate environment "foo" without activating the base
# environment first or having the base's "bin" directory in PATH
source /path/to/condabase/bin/activate foo

# 2) try to run conda in subprocess
python -c '__import__("subprocess").call(["conda", "info"])'
--> OSError: [Errno 2] No such file or directory

What's the recommended most future-proof solution to this? I see there is a $_CONDA_EXE variable, but I can't find it in the documentation. Is it safe to use?

Thanks,
Thomas

[Steve Waterbury]

Post by Thomas Holder

[...] Anaconda Navigator makes extensive use of conda, actually by subprocessing to conda (hey, it might not always be written in python!), and using the --json flag to get machine-friendly structured output. [...]

I also like to do this but found that "conda" is a bash function in conda 4.4 and trying to execute "conda" in a subprocess will fail if working in a (non-base) environment. With conda 4.3, there was actually a "conda" executable in the environment's "bin" directory.
# 1) activate environment "foo" without activating the base
# environment first or having the base's "bin" directory in PATH
source /path/to/condabase/bin/activate foo
# 2) try to run conda in subprocess
python -c '__import__("subprocess").call(["conda", "info"])'
--> OSError: [Errno 2] No such file or directory
What's the recommended most future-proof solution to this? I see there is a $_CONDA_EXE variable, but I can't find it in the documentation. Is it safe to use?

I'll defer to Kale for the 'future-proof' version, but for my use-case
-- i.e. a gui app
installedfrom a conda constructor-created custom installer that includes
conda --
simply using 'conda' as the base command works. If you have a similar
use case,
in which you want to use '--json' to format output, I found by
experimentation
that explcitly creating a Popen process in a separate thread works for
me(note that on
Windows it requires a special flag to avoid having a terminal window appear:
'CREATE_NO_WINDOW' takes care of that).

conda_cmd = ('conda', 'update', '-y', '--json', '-c', '<my channel>',
'<my pkg>')
CREATE_NO_WINDOW = 0x08000000
p = Popen(conda_cmd, stdout=PIPE, bufsize=1,
                  universal_newlines=True, creationflags=CREATE_NO_WINDOW)

... and then get the output from p.stdout; if you are just doing
'info'it's not a problem
to block since the result will be quick, but getting progress
asynchronously for 'install'
or 'update' requires somemore machinations -- I use Queue and another
thread).

HTH,
Steve

[Thomas Holder]

Thomas, the issue here is relevant for you: https://github.com/conda/conda/issues/6810

Thanks, yes that's exactly it. I've subscribed for notifications.

Thomas