<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:#000066;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:#000066;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:#000066;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:#000066;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1030111456;
mso-list-template-ids:440423552;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=white lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>So using the html misses variants?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'> dev-bounces@ensembl.org [mailto:dev-bounces@ensembl.org] <b>On Behalf Of </b>Guillermo Marco Puche<br><b>Sent:</b> 22 May 2013 08:23<br><b>To:</b> dev@ensembl.org<br><b>Subject:</b> Re: [ensembl-dev] VEP variants missing on output<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>Hello Will,<br><br>You was right. I'm getting the 406 variants.<br>I just dropped html in case.<br><br>As always flawless Ensembl support. Thank you !<br><br>Best regards,<br>Guillermo.<br><br>On 05/21/2013 05:13 PM, Will McLaren wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>You get one line of output for each variant/feature overlap, so you will almost always see more output lines than input if you use the default output format. If you use VCF output, you only get one line per variant. <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>You can check how many unique variants there are in the output with e.g.:<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>grep -v # variant_effect_output.txt | cut -f 1 | sort -u | wc -l<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>assuming your variants have unique names.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Try dropping "html" from your config, see if that makes any difference - as the newest feature there, it's got a higher chance of causing problems!<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Will<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On 21 May 2013 16:02, Guillermo Marco Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Hello Will,<br><br>I'm getting more 3000 lines of file output.. this seems really weird....<o:p></o:p></p><pre>wc -l variant_effect_output.txt<o:p></o:p></pre><p class=MsoNormal style='margin-bottom:12.0pt'><b>3936</b><br><br>Here's the way I'm proceeding:<o:p></o:p></p><pre>./<a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a> -i /home/likewise-open/SGNET/gmarco/vep_71_annotation_check/input.vcf -force -fork 4 --database --config vep_71.test<o:p></o:p></pre><p class=MsoNormal><br>Here's the content of vep_71.test:<br><br>dir /home/likewise-open/SGNET/gmarco/.vep<br>toplevel_dir /home/likewise-open/SGNET/gmarco/.vep<br>force_overwrite 1<br>format vcf<br>html 1<br>host 192.19.x.xx<br>port 3306<br>user myuser<br>password mypassword<br>buffer_size 5000 <o:p></o:p></p><div><p class=MsoNormal style='margin-bottom:12.0pt'><br>hgvs 1<br>canonical 1<br>ccds 1<br>check_svs 1<br>domains 1<br>gmaf 1<br>hgnc 1<br>maf_1kg 1<br>numbers 1<br>polyphen b<br>regulatory 1<br>sift b<o:p></o:p></p></div><p class=MsoNormal>Best regards,<br>Guillermo. <o:p></o:p></p><div><div><p class=MsoNormal><br><br>On 05/21/2013 02:30 PM, Will McLaren wrote:<o:p></o:p></p></div></div></div><div><div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>Hi Guillermo, <o:p></o:p></p><div><p class=MsoNormal><br>I'm unable to recreate this, sorry!<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I get 406 going in, 406 coming out every time, whichever combination of those options above I use, and whether I use VCF or standard output.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Here's my run (minus -check_sv):<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal>> perl <a href="http://variant_effect_predictor.pl" target="_blank">variant_effect_predictor.pl</a> -i guill.vcf -vcf -cache -force -fork 4 -hgvs -canon -ccds -domains -gmaf -hgnc -maf_1kg -numbers -poly b -regu -sift b -fasta ~/NFS/Fasta/Homo_sapiens.GRCh37.69.dna.primary_assembly.fa<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Checking/creating FASTA index<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Read existing cache info<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Starting...<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Detected format of input file as vcf<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Read 406 variants into buffer<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:26 - Reading transcript data from cache and/or database<o:p></o:p></p></div><div><p class=MsoNormal>[================================================================] [ 100% ]<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:30 - Retrieved 10891 transcripts (0 mem, 10919 cached, 0 DB, 28 duplicates)<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:30 - Reading regulatory data from cache and/or database<o:p></o:p></p></div><div><p class=MsoNormal>[================================================================] [ 100% ]<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:35 - Retrieved 36955 regulatory features (0 mem, 36955 cached, 0 DB, 0 duplicates)<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:35 - Calculating consequences<o:p></o:p></p></div><div><p class=MsoNormal>[================================================================] [ 100% ]<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:56 - Writing output2013-05-21 13:24:56 - Processed 406 total variants (14 vars/sec, 14 vars/sec total)<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:56 - Wrote stats summary to variant_effect_output.txt_summary.html<o:p></o:p></p></div><div><p class=MsoNormal>2013-05-21 13:24:56 - Finished!<o:p></o:p></p></div><div><p class=MsoNormal>> wc -l variant_effect_output.txt<o:p></o:p></p></div><div><p class=MsoNormal>408<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>It's 408 as it's adding two header lines to the VCF output.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Which 16 are missing from your output, and is it the same 16 each time?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Try writing to a different output file, or on a different disk if you can (perhaps disk space is an issue?)<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Will<o:p></o:p></p></div></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On 21 May 2013 13:15, Guillermo Marco Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal>Hello Will,<br><br>Here's the input: <a href="https://github.com/guillermomarco/vep_plugins_71/blob/master/missing_variants/missing_output_variants.vcf" target="_blank">https://github.com/guillermomarco/vep_plugins_71/blob/master/missing_variants/missing_output_variants.vcf</a><br><br>As you said it's not about the options or plugins. Launching VEP without specyfiying any option still returns an output with missing variants.<br><br>Regards,<br>Guillermo. <o:p></o:p></p><div><div><p class=MsoNormal><br><br><br>On 05/21/2013 01:49 PM, Will McLaren wrote:<o:p></o:p></p></div></div></div><div><div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal>Hi Guillermo, <o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>None of those options should filter out variants.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Are you able to provide any of the files that recreate the problem?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Is there any chance that you are using VCF input and it contains non-variant lines - this would be where the ALT column is empty or "."? If so, this may be your problem. To force these to be included in the output, you should add --allow_non_variant.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Regards<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Will<o:p></o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On 21 May 2013 09:40, Guillermo Marco Puche <<a href="mailto:guillermo.marco@sistemasgenomicos.com" target="_blank">guillermo.marco@sistemasgenomicos.com</a>> wrote:<o:p></o:p></p><div><p class=MsoNormal style='margin-bottom:12.0pt'>Hello,<br><br>I've been checking VEP results, and i've noticed that I'm missing some input variants in the output.<br><br>I think this may be issued to some of the options i'm using to launch vep:<br><br><span style='font-size:7.5pt'>hgvs 1<br>canonical 1<br>ccds 1<br>check_svs 1<br>domains 1<br>gmaf 1<br>hgnc 1<br>maf_1kg 1<br>numbers 1<br>polyphen b<br>regulatory 1<br>sift b</span><br><br>Should be any of these options filtering output? I've disabled all plugins to run this test to be sure that it's not a plugin issue.<o:p></o:p></p><ul type=disc><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1'>With a 406 variant input vcf file, I've missed 16 variants. <o:p></o:p></li><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1'>I then ran VEP with only those 16 missing variants and missed 3 on output. <o:p></o:p></li><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1'>Rerun again and now with 3 missing variants and now not a single one was missing.<o:p></o:p></li></ul><p>I would like to know what's behind that weird behaviour.<o:p></o:p></p><p>Thank you.<o:p></o:p></p><p>Best regards,<br>Guillermo.<o:p></o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div><p class=MsoNormal><br>_______________________________________________<br>Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info</a> <o:p></o:p></p></div></div></blockquote></div></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>_______________________________________________<br>Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><pre>_______________________________________________<o:p></o:p></pre><pre>Dev mailing list <a href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><o:p></o:p></pre><pre>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><o:p></o:p></pre><pre>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><o:p></o:p></pre></blockquote><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>_______________________________________________<br>Dev mailing list <a href="mailto:Dev@ensembl.org">Dev@ensembl.org</a><br>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>Ensembl Blog: <a href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal><br><br><br><o:p></o:p></p><pre>_______________________________________________<o:p></o:p></pre><pre>Dev mailing list <a href="mailto:Dev@ensembl.org">Dev@ensembl.org</a><o:p></o:p></pre><pre>Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev">http://lists.ensembl.org/mailman/listinfo/dev</a><o:p></o:p></pre><pre>Ensembl Blog: <a href="http://www.ensembl.info/">http://www.ensembl.info/</a><o:p></o:p></pre></blockquote><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>-<o:p></o:p></p></div></div></body></html>