Discussion:
[conda] Installing local non-Python conda package to conda env not working
Steve
2018-03-27 20:29:04 UTC
Permalink
I am trying to make a conda env from the source located here:

http://www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz

the contents of which look like this:

$ ls -1 annovar/
annotate_variation.pl
coding_change.pl
convert2annovar.pl
example
humandb
retrieve_seq_from_fasta.pl
table_annovar.pl
variants_reduction.pl


so, in my conda env, I ultimately need to end up with a dir that contains
these files, and it needs to be pre-pended to my $PATH when I activate the
env.

To build a local conda package out of this, I set up my local directory
like this:

$ tree annovar-150617/
annovar-150617/
|-- build.sh
|-- environment.yml
`-- meta.yaml



and the files look like this:

$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
wget http:
//www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
tar xvfz annovar.revision150617.tar.gz
rm -f annovar.revision150617.tar.gz


$ cat annovar-150617/meta.yaml
package:
name: annovar
version: "150617"

source:
url: http:
//www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz

build:
script: build.sh
noarch: generic

outputs:
- name: annovar
version: "150617"
files:
- annovar


$ cat annovar-150617/environment.yml
name: annovar-150617


and I build and install the package with the following command:

source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda install --use-local "$package_path" -n annovar-150617



output:

/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Preparing transaction: done
Verifying transaction: done
Executing transaction: done


When I check that the package is installed in the conda env specified, I
get the following output:

source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}
# packages in environment at /home/kellys04/.conda/envs/annovar-150617:
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10.0/
bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/ibm/
lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/bin:/usr/local/bin:/usr/bin:
/usr/local/sbin:/usr/sbin:/opt/ibutils/bin:/home/kellys04/.local/bin:/home/
kellys04/bin


However, there is a problem here: the path that has been pre-pended to my
$PATH does not actually exist:


$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory


The only thing that has been output there is this 'conda-meta' directory:

$ ls -1 /home/kellys04/.conda/envs/annovar-150617/conda-meta/
annovar-150617-0.json
history


And if I activate the env, the scripts from the source package are not
loaded:

$ source "/shared/miniconda3/bin/activate" annovar-150617 &&
annotate_variation.pl --version
-bash: annotate_variation.pl: command not found



Any ideas? What did I miss?
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+***@continuum.io.
To post to this group, send email to ***@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/e57b358e-8251-4dbb-a483-4b837fbaa169%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Steve
2018-03-27 21:07:58 UTC
Permalink
Thanks, I tried that:

source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda create -c local -n annovar-150617 annovar



output:
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Solving environment: done

## Package Plan ##

environment location: /home/kellys04/.conda/envs/annovar-150617

added / updated specs:
- annovar


The following NEW packages will be INSTALLED:

annovar: 150617-0 local

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate annovar-150617
#
# To deactivate an active environment, use
#
# $ conda deactivate


But the source code is still not being installed in the 'bin' dir shown;

$ source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}


# packages in environment at /home/kellys04/.conda/envs/annovar-150617:
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10.0/
bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/ibm/
lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/bin:/usr/local/bin:/usr/bin:
/usr/local/sbin:/usr/sbin:/opt/ibutils/bin:/home/kellys04/.local/bin:/home/
kellys04/bin

$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory



I did some more investigating and found something that might be related?
There is a 'pkgs' entry in my home directory that appears out of date (from
previous attempts to install)

$ cat /home/kellys04/.conda/pkgs/annovar-150617-0/info/recipe/build.sh
#!/bin/bash
set -x -e
# tar xvfz annovar.revision150617.tar.gz
# && rm -f annovar.revision150617.tar.gz

$ cat /home/kellys04/.conda/pkgs/annovar-150617-0/info/recipe/meta.yaml
# This file created by conda-build 3.7.1
# meta.yaml template originally from:
# /home/kellys04/projects/NGS580-nf/conda/annovar-150617, last modified Tue
Mar 27 03:10:40 2018
# ------------------------------------------------

package:
name: annovar
version: '150617'
source:
url: http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
build:
noarch: generic
string: '0'
extra:
copy_test_source_files: true
final: true




Maybe I need to get rid of this first and try again? Problem is, I am not
sure how; I have already run the following command several times &
re-installed, and the entry still remains:

$ source "/shared/miniconda3/bin/activate" && \
conda remove --name annovar-150617 --all

Remove all packages in environment /home/kellys04/.conda/envs/annovar-150617
:


## Package Plan ##

environment location: /home/kellys04/.conda/envs/annovar-150617


The following packages will be REMOVED:

annovar: 150617-0 local

Proceed ([y]/n)? y
Post by Steve
http://www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
$ ls -1 annovar/
annotate_variation.pl
coding_change.pl
convert2annovar.pl
example
humandb
retrieve_seq_from_fasta.pl
table_annovar.pl
variants_reduction.pl
so, in my conda env, I ultimately need to end up with a dir that contains
these files, and it needs to be pre-pended to my $PATH when I activate the
env.
To build a local conda package out of this, I set up my local directory
$ tree annovar-150617/
annovar-150617/
|-- build.sh
|-- environment.yml
`-- meta.yaml
$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
wget http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
tar xvfz annovar.revision150617.tar.gz
rm -f annovar.revision150617.tar.gz
$ cat annovar-150617/meta.yaml
name: annovar
version: "150617"
url: http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
script: build.sh
noarch: generic
- name: annovar
version: "150617"
- annovar
$ cat annovar-150617/environment.yml
name: annovar-150617
source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda install --use-local "$package_path" -n annovar-150617
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
When I check that the package is installed in the conda env specified, I
source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10.
0/bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/ibm/
/home/kellys04/bin
However, there is a problem here: the path that has been pre-pended to my
$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory
$ ls -1 /home/kellys04/.conda/envs/annovar-150617/conda-meta/
annovar-150617-0.json
history
And if I activate the env, the scripts from the source package are not
$ source "/shared/miniconda3/bin/activate" annovar-150617 &&
annotate_variation.pl --version
-bash: annotate_variation.pl: command not found
Any ideas? What did I miss?
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+***@continuum.io.
To post to this group, send email to ***@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/de22eb87-3266-4bc5-bd88-8fe7a56d5455%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Steve
2018-03-27 21:31:10 UTC
Permalink
Well that successfully removed the entries that were there. However, every
time I try to re-build the package, the same out-of-date versions of
'build.sh' and 'meta.yaml' keep being used. It seems like this file is not
getting updated when I re-build:

$ ll /home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
-rw-rw-r-- 1 kellys04 kellys04 2.1K Mar 27 15:09 /home/kellys04/conda-bld/
noarch/annovar-150617-0.tar.bz2


despite there now being an army of other re-built versions:

$ ll /home/kellys04/conda-bld
total 104K
drwxrwxr-x 26 kellys04 kellys04 4.0K Mar 27 17:22 .
drwx------ 14 kellys04 kellys04 4.0K Mar 27 15:00 ..
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:13 annovar_1522178034453
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:18 annovar_1522178302425
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:18 annovar_1522178337459
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:21 annovar_1522178460108
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:35 annovar_1522179316654
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:35 annovar_1522179331124
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:38 annovar_1522179526558
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:45 annovar_1522179908577
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:45 annovar_1522179926720
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:46 annovar_1522179983874
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:48 annovar_1522180132205
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:08 annovar_1522181336048
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:23 annovar_1522182235847
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:56 annovar_1522184164328
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:58 annovar_1522184335284
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:59 annovar_1522184349306
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:00 annovar_1522184405991
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:11 annovar_1522185068952
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:13 annovar_1522185236242
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:16 annovar_1522185387729
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:22 annovar_1522185732043
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:00 linux-ppc64le
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:09 noarch
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:00 work


this appears relevant to the issue here:
https://github.com/conda/conda-build/issues/727

Is there a way to safely remove all of these old outdated entries? I am not
seeing any "clean" options under 'conda-build'

And if I did clear this cache location, would this supposedly be enough to
get the package to install correctly?
Post by Steve
http://www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
$ ls -1 annovar/
annotate_variation.pl
coding_change.pl
convert2annovar.pl
example
humandb
retrieve_seq_from_fasta.pl
table_annovar.pl
variants_reduction.pl
so, in my conda env, I ultimately need to end up with a dir that contains
these files, and it needs to be pre-pended to my $PATH when I activate the
env.
To build a local conda package out of this, I set up my local directory
$ tree annovar-150617/
annovar-150617/
|-- build.sh
|-- environment.yml
`-- meta.yaml
$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
wget http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
tar xvfz annovar.revision150617.tar.gz
rm -f annovar.revision150617.tar.gz
$ cat annovar-150617/meta.yaml
name: annovar
version: "150617"
url: http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
script: build.sh
noarch: generic
- name: annovar
version: "150617"
- annovar
$ cat annovar-150617/environment.yml
name: annovar-150617
source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda install --use-local "$package_path" -n annovar-150617
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
When I check that the package is installed in the conda env specified, I
source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10.
0/bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/ibm/
/home/kellys04/bin
However, there is a problem here: the path that has been pre-pended to my
$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory
$ ls -1 /home/kellys04/.conda/envs/annovar-150617/conda-meta/
annovar-150617-0.json
history
And if I activate the env, the scripts from the source package are not
$ source "/shared/miniconda3/bin/activate" annovar-150617 &&
annotate_variation.pl --version
-bash: annotate_variation.pl: command not found
Any ideas? What did I miss?
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+***@continuum.io.
To post to this group, send email to ***@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/5ba6bbe3-3618-4617-af61-7db9db63f690%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Paul Hobson
2018-03-27 21:33:14 UTC
Permalink
"conda build purge" I think
Post by Steve
Well that successfully removed the entries that were there. However, every
time I try to re-build the package, the same out-of-date versions of
'build.sh' and 'meta.yaml' keep being used. It seems like this file is not
$ ll /home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
-rw-rw-r-- 1 kellys04 kellys04 2.1K Mar 27 15:09 /home/kellys04/conda-bld/
noarch/annovar-150617-0.tar.bz2
$ ll /home/kellys04/conda-bld
total 104K
drwxrwxr-x 26 kellys04 kellys04 4.0K Mar 27 17:22 .
drwx------ 14 kellys04 kellys04 4.0K Mar 27 15:00 ..
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:13 annovar_1522178034453
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:18 annovar_1522178302425
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:18 annovar_1522178337459
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:21 annovar_1522178460108
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:35 annovar_1522179316654
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:35 annovar_1522179331124
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:38 annovar_1522179526558
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:45 annovar_1522179908577
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:45 annovar_1522179926720
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:46 annovar_1522179983874
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 15:48 annovar_1522180132205
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:08 annovar_1522181336048
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:23 annovar_1522182235847
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:56 annovar_1522184164328
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:58 annovar_1522184335284
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 16:59 annovar_1522184349306
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:00 annovar_1522184405991
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:11 annovar_1522185068952
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:13 annovar_1522185236242
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:16 annovar_1522185387729
drwxrwxr-x 3 kellys04 kellys04 4.0K Mar 27 17:22 annovar_1522185732043
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:00 linux-ppc64le
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:09 noarch
drwxrwxr-x 2 kellys04 kellys04 4.0K Mar 27 15:00 work
this appears relevant to the issue here: https://github.com/conda/
conda-build/issues/727
Is there a way to safely remove all of these old outdated entries? I am
not seeing any "clean" options under 'conda-build'
And if I did clear this cache location, would this supposedly be enough to
get the package to install correctly?
Post by Steve
http://www.openbioinformatics.org/annovar/download/0wgxR2rIV
P/annovar.revision150617.tar.gz
$ ls -1 annovar/
annotate_variation.pl
coding_change.pl
convert2annovar.pl
example
humandb
retrieve_seq_from_fasta.pl
table_annovar.pl
variants_reduction.pl
so, in my conda env, I ultimately need to end up with a dir that contains
these files, and it needs to be pre-pended to my $PATH when I activate the
env.
To build a local conda package out of this, I set up my local directory
$ tree annovar-150617/
annovar-150617/
|-- build.sh
|-- environment.yml
`-- meta.yaml
$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
wget http://www.openbioinformatics.org/annovar/download/0wgxR2rIV
P/annovar.revision150617.tar.gz
tar xvfz annovar.revision150617.tar.gz
rm -f annovar.revision150617.tar.gz
$ cat annovar-150617/meta.yaml
name: annovar
version: "150617"
url: http://www.openbioinformatics.org/annovar/download/0wgxR2rIV
P/annovar.revision150617.tar.gz
script: build.sh
noarch: generic
- name: annovar
version: "150617"
- annovar
$ cat annovar-150617/environment.yml
name: annovar-150617
source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda install --use-local "$package_path" -n annovar-150617
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
When I check that the package is installed in the conda env specified, I
source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10
.0/bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/
/usr/bin:/usr/local/sbin:/usr/sbin:/opt/ibutils/bin:/home/kellys04/.local
/bin:/home/kellys04/bin
However, there is a problem here: the path that has been pre-pended to my
$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory
$ ls -1 /home/kellys04/.conda/envs/annovar-150617/conda-meta/
annovar-150617-0.json
history
And if I activate the env, the scripts from the source package are not
$ source "/shared/miniconda3/bin/activate" annovar-150617 &&
annotate_variation.pl --version
-bash: annotate_variation.pl: command not found
Any ideas? What did I miss?
--
You received this message because you are subscribed to the Google Groups
"conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/
continuum.io/d/msgid/conda/5ba6bbe3-3618-4617-af61-
7db9db63f690%40continuum.io
<https://groups.google.com/a/continuum.io/d/msgid/conda/5ba6bbe3-3618-4617-af61-7db9db63f690%40continuum.io?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+***@continuum.io.
To post to this group, send email to ***@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/CADT3MEADyGwomG2Op1pqeet%3Dv_KtoShwgKkt1wJRkr%2BxMdS_cg%40mail.gmail.com.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Steve
2018-03-27 22:20:20 UTC
Permalink
Ok thanks. To make sure I am working with a clean build environment, I am
using this command between every build attempt:

source "/shared/miniconda3/bin/activate" && \
conda clean -ay && \
conda build purge && \
mv /home/kellys04/conda-bld /home/kellys04/conda-bld$(date +%s) && \
conda remove -y --name annovar-150617 --all


to get rid of everything that might be getting used by accident and forcing
it to re-build completely from the files in the specified directory.

However, I am still having the problem that the source code in the tarball
are not actually being imported into the conda package.

I have tried updating my build files like this:

$ cat annovar-150617/meta.yaml
package:
name: annovar
version: "150617"

source:
url: http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz

build:
noarch: generic

$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
echo $PWD
ls -l


building with this command:

$ ( source "/shared/miniconda3/bin/activate" && conda-build annovar-150617 )


output:

Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for annovar
INFO:conda_build.metadata:Attempting to finalize metadata for annovar
BUILD START: ['annovar-150617-0.tar.bz2']
Source cache directory is: /home/kellys04/conda-bld/src_cache
No hash (md5, sha1, sha256) provided. Source download forced. Add hash to
recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided. Source
download forced. Add hash to recipe to use source cache.
Downloading source to cache: annovar.revision150617.tar.gz
Downloading
http://www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
Success
Extracting download
source tree in: /home/kellys04/conda-bld/annovar_1522188818804/work
+ echo /home/kellys04/conda-bld/annovar_1522188818804/work
/home/kellys04/conda-bld/annovar_1522188818804/work
+ ls -l
total 476
-rwxr-xr-x 1 kellys04 kellys04 202524 Jun 18 2015 annotate_variation.pl
-rwxr-xr-x 1 kellys04 kellys04 13315 Jun 18 2015 coding_change.pl
-rwxrw-rw- 1 kellys04 kellys04 5637 Mar 27 18:13 conda_build.sh
-rwxr-xr-x 1 kellys04 kellys04 165749 Jun 18 2015 convert2annovar.pl
drwxr-xr-x 2 kellys04 kellys04 4096 Jun 18 2015 example
drwxr-xr-x 3 kellys04 kellys04 4096 Jun 18 2015 humandb
-rwxr-xr-x 1 kellys04 kellys04 19121 Jun 18 2015
retrieve_seq_from_fasta.pl
-rwxr-xr-x 1 kellys04 kellys04 33389 Jun 18 2015 table_annovar.pl
-rwxr-xr-x 1 kellys04 kellys04 21003 Jun 18 2015 variants_reduction.pl

Resource usage statistics from building annovar:
Process count: 1
CPU time: unavailable
Memory: 4.2M
Disk usage: 171.3K
Time elapsed: 0:00:02.1

Packaging annovar
INFO:conda_build.build:Packaging annovar
Packaging annovar-150617-0
INFO:conda_build.build:Packaging annovar-150617-0



*No files or script found for output annovarWARNING:conda_build.build:No
files or script found for output annovarnumber of files: 0*Fixing
permissions
updating: annovar-150617-0.tar.bz2
TEST START: /home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Updating index at /home/kellys04/conda-bld/linux-ppc64le to make package
installable with dependencies
INFO:conda_build.build:Updating index at
/home/kellys04/conda-bld/linux-ppc64le to make package installable with
dependencies
Updating index at /home/kellys04/conda-bld/noarch to make package
installable with dependencies
INFO:conda_build.build:Updating index at /home/kellys04/conda-bld/noarch to
make package installable with dependencies
Adding in variants from /tmp/tmpc69rfpdj/info/recipe/conda_build_config.yaml
INFO:conda_build.variants:Adding in variants from
/tmp/tmpc69rfpdj/info/recipe/conda_build_config.yaml
Nothing to test for:
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set. Not uploading wheels: []
####################################################################################
Resource usage summary:

Total time: 0:00:09.8
CPU usage: sys=0:00:00.0, user=0:00:00.0
Maximum memory usage observed: 4.2M
Total disk usage observed (not including envs): 171.3K

Notably, this shows that the source code is being downloaded and extracted
successfully. But none of the files are being included in the final package
output.

I tried including an 'outputs' section in the meta.yaml as per here:
https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#outputs-section

but none of those options work. Any ideas? I am kinda stumped, nothing I've
found in the docs describes anything that helps with this.
Post by Steve
http://www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
$ ls -1 annovar/
annotate_variation.pl
coding_change.pl
convert2annovar.pl
example
humandb
retrieve_seq_from_fasta.pl
table_annovar.pl
variants_reduction.pl
so, in my conda env, I ultimately need to end up with a dir that contains
these files, and it needs to be pre-pended to my $PATH when I activate the
env.
To build a local conda package out of this, I set up my local directory
$ tree annovar-150617/
annovar-150617/
|-- build.sh
|-- environment.yml
`-- meta.yaml
$ cat annovar-150617/build.sh
#!/bin/bash
set -x -e
wget http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
tar xvfz annovar.revision150617.tar.gz
rm -f annovar.revision150617.tar.gz
$ cat annovar-150617/meta.yaml
name: annovar
version: "150617"
url: http://
www.openbioinformatics.org/annovar/download/0wgxR2rIVP/annovar.revision150617.tar.gz
script: build.sh
noarch: generic
- name: annovar
version: "150617"
- annovar
$ cat annovar-150617/environment.yml
name: annovar-150617
source "/shared/miniconda3/bin/activate" && \
package_path="$(conda-build annovar-150617 --output --no-force-upload
--no-anaconda-upload)" && \
echo "$package_path" && \
conda install --use-local "$package_path" -n annovar-150617
/home/kellys04/conda-bld/noarch/annovar-150617-0.tar.bz2
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
When I check that the package is installed in the conda env specified, I
source "/shared/miniconda3/bin/activate" annovar-150617 && \
conda list && \
echo ${PATH} && \
echo ${PYTHONPATH}
#
# Name Version Build Channel
annovar 150617 0 local
/home/kellys04/.conda/envs/annovar-150617/bin:/opt/oss-mldl/bin:/opt/at10.
0/bin:/opt/ibm/lsfsuite/lsf/10.1/linux3.10-glibc2.17-ppc64le/etc:/opt/ibm/
/home/kellys04/bin
However, there is a problem here: the path that has been pre-pended to my
$ ll /home/kellys04/.conda/envs/annovar-150617/bin
ls: cannot access /home/kellys04/.conda/envs/annovar-150617/bin: No such
file or directory
$ ls -1 /home/kellys04/.conda/envs/annovar-150617/conda-meta/
annovar-150617-0.json
history
And if I activate the env, the scripts from the source package are not
$ source "/shared/miniconda3/bin/activate" annovar-150617 &&
annotate_variation.pl --version
-bash: annotate_variation.pl: command not found
Any ideas? What did I miss?
--
You received this message because you are subscribed to the Google Groups "conda - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to conda+***@continuum.io.
To post to this group, send email to ***@continuum.io.
Visit this group at https://groups.google.com/a/continuum.io/group/conda/.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/conda/92ec44b5-2abe-4e8c-a1df-13ba7f6f4cb6%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.
Steve
2018-03-28 01:39:42 UTC
Permalink
This post might be inappropriate. Click to display it.
Loading...