<div dir="ltr">It doesn't cause any problems that I can see, but of course as always please report if you do see any problems.<div><br></div><div>Thanks</div><div><br></div><div>Will</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On 22 May 2013 09:09, Guillermo Marco Puche <span dir="ltr"><<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000066">
<div>It doesn't. I just dropped it because
Will said it could be buggy.<div><div class="h5"><br>
<br>
<br>
On 05/22/2013 10:07 AM, Duarte Molha wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">So
using the html misses variants?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext" lang="EN-US">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext" lang="EN-US"> <a href="mailto:dev-bounces@ensembl.org" target="_blank">dev-bounces@ensembl.org</a>
[<a href="mailto:dev-bounces@ensembl.org" target="_blank">mailto:dev-bounces@ensembl.org</a>] <b>On Behalf Of </b>Guillermo
Marco Puche<br>
<b>Sent:</b> 22 May 2013 08:23<br>
<b>To:</b> <a href="mailto:dev@ensembl.org" target="_blank">dev@ensembl.org</a><br>
<b>Subject:</b> Re: [ensembl-dev] VEP variants missing
on output<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hello Will,<br>
<br>
You was right. I'm getting the 406 variants.<br>
I just dropped html in case.<br>
<br>
As always flawless Ensembl support. Thank you !<br>
<br>
Best regards,<br>
Guillermo.<br>
<br>
On 05/21/2013 05:13 PM, Will McLaren wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">You get one line of output for each
variant/feature overlap, so you will almost always see
more output lines than input if you use the default output
format. If you use VCF output, you only get one line per
variant. <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">You can check how many unique
variants there are in the output with e.g.:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">grep -v # variant_effect_output.txt |
cut -f 1 | sort -u | wc -l<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">assuming your variants have unique
names.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Try dropping "html" from your config,
see if that makes any difference - as the newest feature
there, it's got a higher chance of causing problems!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Will<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 21 May 2013 16:02, Guillermo Marco
Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>>
wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hello
Will,<br>
<br>
I'm getting more 3000 lines of file output.. this
seems really weird....<u></u><u></u></p>
<pre>wc -l variant_effect_output.txt<u></u><u></u></pre>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b>3936</b><br>
<br>
Here's the way I'm proceeding:<u></u><u></u></p>
<pre>./<a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a> -i /home/likewise-open/SGNET/gmarco/vep_71_annotation_check/input.vcf -force -fork 4 --database --config vep_71.test<u></u><u></u></pre>
<p class="MsoNormal"><br>
Here's the content of vep_71.test:<br>
<br>
dir
/home/likewise-open/SGNET/gmarco/.vep<br>
toplevel_dir
/home/likewise-open/SGNET/gmarco/.vep<br>
force_overwrite 1<br>
format vcf<br>
html 1<br>
host 192.19.x.xx<br>
port 3306<br>
user myuser<br>
password mypassword<br>
buffer_size 5000 <u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
hgvs 1<br>
canonical 1<br>
ccds 1<br>
check_svs 1<br>
domains 1<br>
gmaf 1<br>
hgnc 1<br>
maf_1kg 1<br>
numbers 1<br>
polyphen b<br>
regulatory 1<br>
sift b<u></u><u></u></p>
</div>
<p class="MsoNormal">Best regards,<br>
Guillermo. <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
On 05/21/2013 02:30 PM, Will McLaren wrote:<u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi Guillermo, <u></u><u></u></p>
<div>
<p class="MsoNormal"><br>
I'm unable to recreate this, sorry!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I get 406 going in, 406
coming out every time, whichever combination
of those options above I use, and whether I
use VCF or standard output.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Here's my run (minus
-check_sv):<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">> perl <a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a>
-i guill.vcf -vcf -cache -force -fork 4
-hgvs -canon -ccds -domains -gmaf -hgnc
-maf_1kg -numbers -poly b -regu -sift b
-fasta
~/NFS/Fasta/Homo_sapiens.GRCh37.69.dna.primary_assembly.fa<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Checking/creating FASTA index<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Read existing cache info<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Starting...<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Detected format of input file as vcf<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Read 406 variants into buffer<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:26 -
Reading transcript data from cache and/or
database<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[================================================================]
[ 100% ]<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:30 -
Retrieved 10891 transcripts (0 mem, 10919
cached, 0 DB, 28 duplicates)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:30 -
Reading regulatory data from cache and/or
database<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[================================================================]
[ 100% ]<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:35 -
Retrieved 36955 regulatory features (0
mem, 36955 cached, 0 DB, 0 duplicates)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:35 -
Calculating consequences<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">[================================================================]
[ 100% ]<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:56 -
Writing output2013-05-21 13:24:56 -
Processed 406 total variants (14 vars/sec,
14 vars/sec total)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:56 -
Wrote stats summary to
variant_effect_output.txt_summary.html<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2013-05-21 13:24:56 -
Finished!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">> wc -l
variant_effect_output.txt<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">408<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It's 408 as it's adding
two header lines to the VCF output.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Which 16 are missing
from your output, and is it the same 16
each time?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Try writing to a
different output file, or on a different
disk if you can (perhaps disk space is an
issue?)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Will<u></u><u></u></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 21 May 2013 13:15,
Guillermo Marco Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>>
wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Hello Will,<br>
<br>
Here's the input: <a href="https://github.com/guillermomarco/vep_plugins_71/blob/master/missing_variants/missing_output_variants.vcf" target="_blank">https://github.com/guillermomarco/vep_plugins_71/blob/master/missing_variants/missing_output_variants.vcf</a><br>
<br>
As you said it's not about the options
or plugins. Launching VEP without
specyfiying any option still returns an
output with missing variants.<br>
<br>
Regards,<br>
Guillermo. <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
<br>
On 05/21/2013 01:49 PM, Will McLaren
wrote:<u></u><u></u></p>
</div>
</div>
</div>
<div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi Guillermo, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">None of those
options should filter out
variants.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Are you able
to provide any of the files that
recreate the problem?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Is there any
chance that you are using VCF
input and it contains
non-variant lines - this would
be where the ALT column is empty
or "."? If so, this may be your
problem. To force these to be
included in the output, you
should add --allow_non_variant.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Regards<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Will<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 21 May
2013 09:40, Guillermo Marco
Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>>
wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hello,<br>
<br>
I've been checking VEP
results, and i've noticed that
I'm missing some input
variants in the output.<br>
<br>
I think this may be issued to
some of the options i'm using
to launch vep:<br>
<br>
<span style="font-size:7.5pt">hgvs
1<br>
canonical 1<br>
ccds 1<br>
check_svs 1<br>
domains 1<br>
gmaf 1<br>
hgnc 1<br>
maf_1kg 1<br>
numbers 1<br>
polyphen b<br>
regulatory 1<br>
sift b</span><br>
<br>
Should be any of these options
filtering output? I've
disabled all plugins to run
this test to be sure that it's
not a plugin issue.<u></u><u></u></p>
<ul type="disc">
<li class="MsoNormal">With a 406
variant input vcf file, I've
missed 16 variants. <u></u><u></u></li>
<li class="MsoNormal">I then ran VEP
with only those 16 missing
variants and missed 3 on
output. <u></u><u></u></li>
<li class="MsoNormal">Rerun again and
now with 3 missing variants
and now not a single one was
missing.<u></u><u></u></li>
</ul>
<p>I would like to know what's
behind that weird behaviour.<u></u><u></u></p>
<p>Thank you.<u></u><u></u></p>
<p>Best regards,<br>
Guillermo.<u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and
subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info</a>
<u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe
info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><br>
<br>
<u></u><u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><u></u><u></u></pre>
<pre>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><u></u><u></u></pre>
<pre>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><u></u><u></u></pre>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<u></u><u></u></p>
<pre>_______________________________________________<u></u><u></u></pre>
<pre>Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><u></u><u></u></pre>
<pre>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><u></u><u></u></pre>
<pre>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><u></u><u></u></pre>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">-<u></u><u></u></p>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a>
</pre>
</blockquote>
</div></div></div>
<br>_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br></blockquote></div><br></div>