<div dir="ltr">I'm wondering if 5.8.8 has different regex handling to newer Perl versions. Someone else in the Ensembl team may know better than me on this one.<div><br></div><div>I believe the Ensembl project now recommends at least 5.10 (according to <a href="http://www.ensembl.org/info/docs/api/api_installation.html">http://www.ensembl.org/info/docs/api/api_installation.html</a> at least); most people in the wild use 5.14 or 5.16 AFAIK.</div><div><br></div><div>If you can possibly try a newer version of Perl this may solve your issues. Perlbrew is a nice way to manage different versions and module sets <a href="http://perlbrew.pl/">http://perlbrew.pl/</a></div><div><br></div><div>Will</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 4 June 2015 at 17:02, Guillermo Marco Puche <span dir="ltr"><<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF">
Hello Will,<br>
<br>
Wrong behavior machine has Centos 5.4 and Perl v5.8.8 built for
x86_64-linux-thread-multi.<br>
<br>
So should I completly remove ?<span class=""><br>
<pre><b> foreach my $line(split /\r|\R/) {</b></pre>
<br></span>
I was thinking about just removing \R from regex.<br>
<br>
Regards,<br>
Guillermo.<div><div class="h5"><br>
<br>
<div>On 04/06/15 17:49, Will McLaren wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Thanks
<div><br>
</div>
<div>I had forgotten about that change. You could just edit the
script and change or even remove the regexp:</div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
<div><span style="white-space:pre-wrap">foreach my $line(($_)) {</span><br>
</div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
<div><span style="white-space:pre-wrap">What's your Perl version
and system architecture? I'm surprised this has not caught
anyone else out.</span></div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
<div><span style="white-space:pre-wrap">Will</span></div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 4 June 2015 at 14:47, Guillermo
Marco Puche <span dir="ltr"><<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF"> Hi Will,<br>
<br>
I've been comparing variant_effect_predictor script from
version 75 vs 79.<br>
After adding a few prints to VEP.pm inside I've spotted
the bug. However I cannot resolve it.<br>
<br>
Those lanes are new from 75 to 79 in VEP script (175 and
176):<br>
<b><br>
</b>
<pre><b> # split again to avoid Windows character nonsense</b></pre>
<pre><b> foreach my $line(split /\r|\R/) {</b></pre>
<br>
<br>
I've checked that script is spliting line each time it
finds a capital R in VCF file as identifying it as a
newline character from Windows. I can't reproduce it in
virtual machine since its a fresh Linux install. In my
work environment I'm getting this kind of bug, so I guess
it has something to do with file enconding or locale? Has
anyone else experienced this?<br>
<br>
Now I know where's the error but I've no idea how to solve
it.<br>
<br>
Regards,<br>
Guillermo.
<div>
<div><br>
<br>
<div>On 04/06/15 15:16, Will McLaren wrote:<br>
</div>
<blockquote type="cite">
<p dir="ltr">Sorry Guillermo, I'm running out of
ideas. </p>
<p dir="ltr">Does the test unit run OK?</p>
<p dir="ltr">perl
ensembl-tools/scripts/variant_effect_predictor/t/variant_effect_predictor.t</p>
<p dir="ltr">Will</p>
<div class="gmail_quote">On 4 Jun 2015 12:27,
"Guillermo Marco Puche" <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF"> Hi Will,<br>
<br>
I'm getting the exact same error with
example_GRCh37.vcf:<br>
<pre>ERROR: Could not detect input file format
</pre>
<br>
I've made a test script as you suggest with
the following code and I don't get any error:<br>
<br>
<pre>#!/usr/bin/env perl
use strict;
use Bio::EnsEMBL::Variation::Utils::VEP qw(detect_format);</pre>
<br>
Regards,<br>
Guillermo.<br>
<br>
<div>On 04/06/15 13:12, Will McLaren wrote:<br>
</div>
<blockquote type="cite">
<p dir="ltr">Hi again</p>
<p dir="ltr">If the script is not detecting
the input format then it is almost
certainly an issue with the input file.
There's very little code that gets run to
detect the format, and it's all internal
to the VEP code.</p>
<p dir="ltr">You could write a short script
to test the method, just import
detect_format from
Bio:EnsEMBL::Variation::Utils::VEP</p>
<p dir="ltr">Does it detect the
example_GRCh37.vcf format correctly?</p>
<p dir="ltr">The file you shared on Dropbox
works fine for me on my Mac and a Linux
box. </p>
<p dir="ltr">Will</p>
<div class="gmail_quote">On 4 Jun 2015
10:44, "Guillermo Marco Puche" <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF">
Hi again Will,<br>
<br>
I'm trying with latest ensembl 80.<br>
If I don't specify <b>-format vcf</b>
I get the following error:<br>
<br>
<pre>perl ensembl-tools/scripts/variant_effect_predictor/<a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a> -i input.vcf -database --force_overwrite
2015-06-04 11:36:59 - Starting...
ERROR: Could not detect input file format</pre>
<br>
If I force format with<b> -format vcf
</b>I get all the errors. (see error
log attached). I'm using the same
input.vcf file I posted yesterday.<br>
Just to discard it's not VCF, I've
installed a fresh linux on virtual
machine and just cloned and setup
Ensembl and Bioperl. On fresh Linux
install I was only asked to install
MySQL perl module (I installed it via
CPAN).<br>
It's working like a cake. <br>
<br>
I discard there's a problem with the
input VCF because I'm using exactly
the input over the two environments
(and the same one you used to test it
yesterday) <br>
<br>
So my question is: Does VEP script use
any other library, environment
variable or tool which may be
interfering?<br>
<br>
Best regards,<br>
Guillermo.<br>
<br>
<br>
<div>On 04/06/15 09:32, Guillermo
Marco Puche wrote:<br>
</div>
<blockquote type="cite"> Hi again
Will,<br>
<br>
I've completly cleaned PERL5LIB
environment var. I've been testing
changing between bioperl 1.2.3 and
bioperl 1.6.1 and got same
warnings/errors.<br>
I've cloned again all 79 API like
you suggested in a new tmp location
and included it in $PERL5LIB.<br>
<br>
<pre><b>echo $PERL5LIB </b></pre>
/share/apps/local/bioperl-live:/share/gluster/tests/gmarco/tmp/ensembl/modules:/share/gluster/tests/gmarco/tmp/ensembl-funcgen/modules:/share/gluster/tests/gmarco/tmp/ensembl-variation/modules<br>
<br>
<pre><b>ll /share/gluster/tests/gmarco/tmp</b></pre>
total 20<br>
drwxrwxr-x 8 gmarco users 4096 jun
4 08:44 ensembl<br>
drwxrwxr-x 8 gmarco users 146 jun
4 08:46 ensembl-funcgen<br>
drwxrwxr-x 5 gmarco users 64 jun
4 08:43 ensembl-tools<br>
drwxrwxr-x 10 gmarco users 4096 jun
4 08:45 ensembl-variation<br>
<br>
<pre><b>perl ensembl-tools/scripts/variant_effect_predictor/<a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a> -i input.vcf -database --force_overwrite</b></pre>
2015-06-04 09:29:13 - Starting...<br>
ERROR: Could not detect input file
format<br>
<br>
If use the following flags <b>-format
vcf</b> <b>-vcf </b>then I start
getting all those errors (see
yesterday log).<br>
<br>
Is there any other Perl lib or
requirement I could be missing? As I
said it's very weird I have 0
problems with Ensembl 75 local API.<br>
<br>
Best regards,<br>
Guillermo.<br>
<br>
<div>On 03/06/15 18:14, Will McLaren
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi again,
<div><br>
</div>
<div>I can't recreate the
problem with that input file
I'm afraid, either on my
normal setup or scrubbing
PERL5LIB and starting from
scratch.</div>
<div><br>
</div>
<div>See commands I used and
input below.</div>
<div><br>
</div>
<div>Perhaps you haven't got
release/79 of ensembl-tools
too?</div>
<div><br>
</div>
<div>Have you tried running the
installer from within
ensembl-tools/scripts/variant_effect_predictor?
This shouldn't affect your
PERL5LIB or other git
checkouts.</div>
<div><br>
</div>
<div>Will</div>
<div><br>
</div>
<div>===================</div>
<div><br>
</div>
<div>mkdir ~/src/tmp</div>
<div>cd ~/src/tmp</div>
<div>git clone --branch
release/79 <a href="https://github.com/Ensembl/ensembl-tools.git" target="_blank">https://github.com/Ensembl/ensembl-tools.git</a><br>
</div>
<div>git clone --branch
release/79 <a href="https://github.com/Ensembl/ensembl.git" target="_blank">https://github.com/Ensembl/ensembl.git</a></div>
<div>git clone --branch
release/79 <a href="https://github.com/Ensembl/ensembl-variation.git" target="_blank">https://github.com/Ensembl/ensembl-variation.git</a><br>
</div>
<div>git clone --branch
release/79 <a href="https://github.com/Ensembl/ensembl-funcgen.git" target="_blank">https://github.com/Ensembl/ensembl-funcgen.git</a><br>
</div>
<div>export
PERL5LIB=ensembl/modules:ensembl-variation/modules:ensembl-funcgen/modules:/Users/will/src/bioperl-1.2.3/:/Users/will/src/lib/perl5/<br>
</div>
<div>
<div>perl
ensembl-tools/scripts/variant_effect_predictor/<a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a>
-i ~/Downloads/input.vcf
-database</div>
<div>2015-06-03 17:09:54 -
Starting...</div>
<div>2015-06-03 17:09:54 -
Detected format of input
file as vcf</div>
<div>2015-06-03 17:09:54 -
Read 1 variants into buffer</div>
<div>2015-06-03 17:09:54 -
Reading transcript data from
cache and/or database</div>
<div>[================================================================================================================================]
[ 100% ]</div>
<div>2015-06-03 17:10:00 -
Retrieved 7 transcripts (0
mem, 0 cached, 7 DB, 0
duplicates)</div>
<div>2015-06-03 17:10:00 -
Analyzing chromosome 1</div>
<div>2015-06-03 17:10:00 -
Analyzing variants</div>
<div>[================================================================================================================================]
[ 100% ]</div>
<div>2015-06-03 17:10:00 -
Calculating consequences</div>
<div>2015-06-03 17:10:00 -
Processed 1 total variants
(0 vars/sec, 0 vars/sec
total)</div>
<div>2015-06-03 17:10:00 -
Wrote stats summary to
variant_effect_output.txt_summary.html</div>
<div>2015-06-03 17:10:00 -
Finished!</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 3
June 2015 at 16:51, Guillermo
Marco Puche <span dir="ltr"><<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF"> Hi
Will,<br>
<br>
I've been checking and I
can't see any unintended
whitespace or problem with
tabulations.<br>
I've no issues with old
vep 75 script and API.
I've updated the Bioperl
lib in $PERL5LIB variable
from 1.2.3 to 1.6.1 (I
didn't see this change
before sorry) however I'm
still getting all those
errors.<br>
<br>
Here's a link where you
can download the VCF I'm
using as input: <a href="https://www.dropbox.com/sh/felwyoo5kl2mgty/AAC177Digqy-_mEmyk9WvmYba/input.vcf?dl=0" target="_blank">https://www.dropbox.com/sh/felwyoo5kl2mgty/AAC177Digqy-_mEmyk9WvmYba/input.vcf?dl=0</a><br>
<br>
Thank you.<br>
<br>
Best regards,<br>
Guille.
<div>
<div><br>
<br>
<div>On 03/06/15
17:30, Will McLaren
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi
Guille,
<div><br>
It looks to me
like your input
is not being
parsed properly.<br>
</div>
<div><br>
</div>
<div>Check the
formatting of
your input VCF;
double check
that it is valid
VCF, and that
you haven't got
any unintended
whitespace on
any of the
lines.</div>
<div><br>
</div>
<div>If you still
have an issue,
can you send a
line or two of
the input that
recreates these
issues?</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
<div>Will McLaren</div>
<div>Ensembl
Variation</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
3 June 2015 at
16:16, Guillermo
Marco Puche <span dir="ltr"><<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000066" bgcolor="#FFFFFF"> Dear devs,<br>
<br>
I'm trying
ensembl 79
VEP.<br>
<br>
This is my
dummy input
VCF: <a href="http://pastebin.com/kFKWH50q#" target="_blank">http://pastebin.com/kFKWH50q#</a><br>
<br>
I've cloned
and installed
API from
github as
always (this
step is
repeated for
variaton,
funcgen and
compara):<br>
<ul>
<li>git clone
--branch
release/79 <a href="https://github.com/Ensembl/ensembl.git" style="word-wrap:break-word;text-decoration:underline;color:rgb(92,53,102)" target="_blank">https://github.com/Ensembl/ensembl.git</a> ensembl_79</li>
</ul>
<p>PERL5LIB
env variable
is correctly
pointing to
the cloned
API: <br>
</p>
<ul>
<li>echo
$PERL5LIB <br>
/share/apps/local/bioperl-live:/share/apps/src/ensembl_79/modules:/share/apps/src/ensembl_79-compara/modules:/share/apps/src/ensembl_79-variation/modules:/share/apps/src/ensembl_79-functgenomics/modules</li>
</ul>
<p>However I'm
getting a lot
of errors I
really don't
understand. It
seems like a
bug with API
installation
with me. If I
change
$PERL5LIB
variable to
point to 75
API (previous
version I was
using) I can't
reproduce the
errors VEP
script works
for this old
75 version.<br>
</p>
<p>I've been
reading the
docs again and
I can't seen
any additional
PERL library
requirement.<br>
</p>
<p>Here's the
error log: <a href="http://pastebin.com/VvQrkEQZ" target="_blank">http://pastebin.com/VvQrkEQZ</a><br>
</p>
<p><br>
Thank you!<br>
</p>
<p>Best
regards,<br>
Guille.<br>
</p>
</div>
<br>
_______________________________________________<br>
Dev mailing
list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting
guidelines and
subscribe/unsubscribe
info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog:
<a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
<br>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and
subscribe/unsubscribe info:
<a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
<br>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
<br>
<br>
</div>
<br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and
subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
</div>
<br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe
info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br></blockquote></div><br></div>