<div dir="ltr">Hi Alex,<div><br></div><div>Are you using the latest version of VEP? We have in version 82 an alternative FASTA indexer that is more robust to index corruption issues; it is also faster and allows you to use bgzipped FASTA files which take up much less disk space.</div><div><br></div><div><a href="http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html">http://www.ensembl.org/info/docs/tools/vep/script/vep_download.html</a><br></div><div><br></div><div>Regards</div><div><br></div><div>Will McLaren</div><div>Ensembl Variation</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 22 October 2015 at 07:39, Alex Beesley <span dir="ltr"><<a href="mailto:Alex.Beesley@telethonkids.org.au" target="_blank">Alex.Beesley@telethonkids.org.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Konrad<br>
<br>
Yes, I completely deleted the fasta index and let VEP recreate it (running<br>
to completion), but the problem persists. I even scoured my file system<br>
for other possible copies of the fasta and index, and specifically pointed<br>
the VEP script at the explicit fasta file, but the issue persists. The<br>
ONLY way I can generate the apparently correct LoF calls for these sites<br>
is to use VEP in online mode. Even wierder, I noticed that when running in<br>
offline mode, a small number of the sites flip between LC and HC from run<br>
to run!! (though the majority still remain LC and thus incorrect).<br>
<br>
<br>
These incorrect LC calls are essentially what I get if I try running VEP<br>
offline WITHOUT any fasta file. Hence, it seems as though, although VEP is<br>
correctly using the local fasta file (i.e. It looks for the file, detects<br>
it, creates the index), the LoF plugin is not recognising the local fasta<br>
file (i.e. The LoF output is the same as if I didn¹t even use a fasta<br>
file). To refresh I am using a GRChr37 homo_sapiens cache (NOT<br>
homo_sapiens_merged and NOT homo_sapiens_refeq). And I have tried with<br>
both a compressed and uncompressed fasta file options.<br>
<br>
All ideas welcome :)<br>
<br>
Cheers<br>
alex<br>
<br>
On 16/10/2015 11:58 pm, "<a href="mailto:dev-bounces@ensembl.org">dev-bounces@ensembl.org</a> on behalf of<br>
<a href="mailto:dev-request@ensembl.org">dev-request@ensembl.org</a>" <<a href="mailto:dev-bounces@ensembl.org">dev-bounces@ensembl.org</a> on behalf of<br>
<a href="mailto:dev-request@ensembl.org">dev-request@ensembl.org</a>> wrote:<br>
<br>
>------------------------------<br>
><br>
>Message: 2<br>
>Date: Fri, 16 Oct 2015 11:57:43 -0400<br>
>From: Konrad Karczewski <<a href="mailto:konradk@broadinstitute.org">konradk@broadinstitute.org</a>><br>
>Subject: Re: [ensembl-dev] Loftee and VCFCols Difficulties via port<br>
> 3337<br>
>To: Ensembl developers list <<a href="mailto:dev@ensembl.org">dev@ensembl.org</a>><br>
>Message-ID: <<a href="mailto:D18E3FB8-4CF2-4C7E-BF6D-3328E969C72E@broadinstitute.org">D18E3FB8-4CF2-4C7E-BF6D-3328E969C72E@broadinstitute.org</a>><br>
>Content-Type: text/plain; charset="windows-1252"<br>
><br>
>Hello!<br>
><br>
>Right, I understand fasta is not required offline for VEP, but it is for<br>
>LOFTEE (we dig into it to check splice site sequence). Just let me<br>
>confirm: are you deleting the index, letting VEP recreate it AND letting<br>
>VEP complete its run? (Not just the index creation but going all the way<br>
>to "Finished!") This tripped me up for the longest time. One hacky<br>
>workaround I've found for this is to delete the index and then remove<br>
>write permissions on the directory the FASTA lives in, which will force<br>
>VEP to recreate the index every time. Not the most efficient, but I do<br>
>that on my end for various reasons.<br>
><br>
>-Konrad<br>
><br>
>>On Oct 12, 2015, at 2:03 AM, Alex Beesley<br>
>><<a href="mailto:Alex.Beesley@telethonkids.org.au">Alex.Beesley@telethonkids.org.au</a>> wrote:<br>
>>Hi Konrad & Will<br>
>>Thanks for both your comments/suggestions re the loftee plugin.<br>
>>Unfortunately I have been UNABLE to fix the problem by reinstalling or<br>
>>recreating the fasta index.<br>
>>In fact the LoF call problem persists if you run offline without using a<br>
>>fasta file at all (a fasta file is not required offline).<br>
>>FYI I actually had considerable trouble using VEP at all with the GRChr37<br>
>>fasta downloaded for ensemble-82 via the API, and in the end had to use<br>
>>the ?NO_HTSLIB flag. With that in place<br>
>>VEP will run OK offline using the local fasta file but the LoF calls are<br>
>>still wrong (LC instead of HC). Same deal if I delete the index and let<br>
>>VEP recreate it.<br>
>>Clearly there is some difference between the databases that are<br>
>>downloaded<br>
>>via the API for GRChr37 vs what is online.<br>
>>This seems like it could have more serious ramifications for other uses<br>
>>as<br>
>>well (i.e. It may not just be a problem with the LoF plugin??).<br>
>>I?ve confirmed the error using both VEP78 and VEP81 with loftee too.<br>
>>Any suggestions welcome!<br>
>>Cheers<br>
>>Alex<br>
>>>------------------------------<br>
>>>Message: 2<br>
>>>Date: Fri, 09 Oct 2015 12:44:30 -0700 (PDT)<br>
>>>From: "Konrad Karczewski" <<a href="mailto:konradk@broadinstitute.org">konradk@broadinstitute.org</a>><br>
>>>Subject: Re: [ensembl-dev] FW: Loftee and VCFCols Difficulties via<br>
>>> port 3337<br>
>>>To: "Ensembl developers list" <<a href="mailto:dev@ensembl.org">dev@ensembl.org</a>><br>
>>>Message-ID: <1444419869904.9698efa2@Nodemailer><br>
>>>Content-Type: text/plain; charset="utf-8"<br>
>>>Hi all,<br>
>>>Developer of LOFTEE here - I've seen this kind of thing before (Issue<br>
>>>#2). The issue is actually with the FASTA index file created by<br>
>>>VEP/BioPerl. When you're in online mode, it's getting the right sequence<br>
>>>of the splice site, but when offline with a malformed index, it always<br>
>>>returns NN resulting in many NON_CAN_SPLICE and NON_CAN_SPLICE_SURR<br>
>>>annotations.<br>
>>>I suggest deleting<br>
>>>the?Homo_sapiens.GRCh37.75.dna.primary_assembly.fa.index file and<br>
>>>recreating it: to do this, just run VEP on a small test file. Important<br>
>>>note: you must let VEP run to completion, even though Checking/Creating<br>
>>>FASTA Index is near the beginning and it starts writing one at that<br>
>>>time,<br>
>>>it can be a corrupt index if you cancel it at that point. I typically<br>
>>>just annotate a single variant so it finishes quickly. Don't ask how I<br>
>>>figured all this out...<br>
>>>Hope that helps!<br>
>>>-Konrad<br>
>>>On Fri, Oct 9, 2015 at 4:40 AM, Will McLaren <<a href="mailto:wm2@ebi.ac.uk">wm2@ebi.ac.uk</a>> wrote:<br>
>>>>Hi Alex,<br>
>>>>Regarding issue 1, have you considered using VCF output instead of the<br>
>>>>default tab-delimited output?<br>
>>>><a href="http://www.ensembl.org/info/docs/tools/vep/vep_formats.html#vcfout" rel="noreferrer" target="_blank">http://www.ensembl.org/info/docs/tools/vep/vep_formats.html#vcfout</a><br>
>>>>Have you tried contacting the VAX authors? Michael Yourshaw is usually<br>
>>>>very<br>
>>>>responsive when I have communicated with him in the past.<br>
>>>>I'm sure you can appreciate we have to prioritise debugging and fixing<br>
>>>>our<br>
>>>>own code, but please do get back to us if you still have any<br>
>>>>outstanding<br>
>>>>issues.<br>
>>>>You may also like to try another available LoF plugin, LOFTEE from<br>
>>>>Daniel<br>
>>>>MacArthur's lab: <a href="https://github.com/konradjk/loftee" rel="noreferrer" target="_blank">https://github.com/konradjk/loftee</a><br>
>>>>Regards<br>
>>>>Will McLaren<br>
>>>>Ensembl Variation<br>
>>>>On 9 October 2015 at 03:22, Alex Beesley<br>
>>>><<a href="mailto:Alex.Beesley@telethonkids.org.au">Alex.Beesley@telethonkids.org.au</a>><br>
>>>>wrote:<br>
>>>>>Dear Team<br>
>>>>>I am experiencing significant difficulties with both the LoF.pm and<br>
>>>>>VCFCols.pm plugins with VEP (FYI I am using a GRCh37 cache downloaded<br>
>>>>>using<br>
>>>>>the installer script and default settings (ensembl-tools release-82)).<br>
>>>>># Issue 1<br>
>>>>>I want to use VCFCols.pm in order to obtain the original REF and ALT<br>
>>>>>alleles from the VCF (to aid with interpretation of complex variants).<br>
>>>>>However it seems that the only way to run VCFCols.pm plugin is in the<br>
>>>>>online mode ? if one tries to run it in offline mode (see first code<br>
>>>>>example below), VEP returns an error relating to<br>
>>>>>"$config->{ga}->fetch_by_transcript_stable_id($transcript_id)?.<br>
>>>>>However,<br>
>>>>>when running online (see second code example), it is extremely slow.<br>
>>>>>This<br>
>>>>>is incredibly frustrating because I do not wish to use any of the VAX<br>
>>>>>functionality or its related databases, I simply wish to grab the<br>
>>>>>original<br>
>>>>>REF, ALT and other VCF column headers (including the genotypes and<br>
>>>>>FORMAT<br>
>>>>>fields) in my VEP output. Is there another way to grab the original<br>
>>>>>VCF<br>
>>>>>columns in the VEP output other than using VCFCols.pm? Or a way to<br>
>>>>>modify<br>
>>>>>the plugin such that it can work offline?<br>
>>>>>perl ${VEP}/<a href="http://variant_effect_predictor.pl" rel="noreferrer" target="_blank">variant_effect_predictor.pl</a> -i ${INPUT_VCF} -o<br>
>>>>>${INPUT_VCF%*.vcf}.vep --cache --assembly GRCh37 --offline \<br>
>>>>> --force_overwrite --check_existing --fork 24 \<br>
>>>>> --everything --flag_pick \<br>
>>>>> --plugin CADD,${CADD_SNV},${CADD_INDEL} \<br>
>>>>> --plugin ExAC,${EXAC} \<br>
>>>>>?-plugin VCFCols \<br>
>>>>> --plugin<br>
>>>>>LoF,human_ancestor_fa:/home/san/alex/.vep/Plugins/loftee-master/human_<br>
>>>>>an<br>
>>>>>cestor.fa.gz<br>
>>>>>\<br>
>>>>> --fields<br>
>>>>>Uploaded_variation,Location,REF,ALT,INFO,FORMAT,LoF,LoF_filter,LoF_fla<br>
>>>>>gs<br>
>>>>>,CADD_RAW,CADD_PHRED,ExAC_AF<br>
>>>>>perl ${VEP}/<a href="http://variant_effect_predictor.pl" rel="noreferrer" target="_blank">variant_effect_predictor.pl</a> -i ${INPUT_VCF} -o<br>
>>>>>${INPUT_VCF%*.vcf}.ONLINE.vep --cache --assembly GRCh37 --port 3337 \<br>
>>>>> --force_overwrite --check_existing --fork 24 \<br>
>>>>> --everything --flag_pick \<br>
>>>>> --plugin CADD,${CADD_SNV},${CADD_INDEL} \<br>
>>>>> --plugin ExAC,${EXAC} \<br>
>>>>>?-plugin VCFCols \<br>
>>>>> --plugin<br>
>>>>>LoF,human_ancestor_fa:/home/san/alex/.vep/Plugins/loftee-master/human_<br>
>>>>>an<br>
>>>>>cestor.fa.gz<br>
>>>>>\<br>
>>>>> --fields<br>
>>>>>Uploaded_variation,Location,REF,ALT,INFO,FORMAT,LoF,LoF_filter,LoF_fla<br>
>>>>>gs<br>
>>>>>,CADD_RAW,CADD_PHRED,ExAC_AF<br>
>>>>># Issue 2<br>
>>>>>When running VEP in either of the two modes shown above, I obtain<br>
>>>>>different confidence calls from the LoF.pm in regards to frameshift<br>
>>>>>mutations. Specifically, for the example shown below, the LoF.pm<br>
>>>>>plugin<br>
>>>>>will call the variant HC (high confidence) in ONLINE mode, but LC (low<br>
>>>>>confidence) when running offline. The particular flag thrown up for<br>
>>>>>the LC<br>
>>>>>call relates to non-canonical intron splice sites, however I have<br>
>>>>>checked<br>
>>>>>this particular variant on UCSC and the splice appear to be canonical,<br>
>>>>>thus<br>
>>>>>the ONLINE vep output is correct, and the offline appears to be<br>
>>>>>incorrect.<br>
>>>>>Since I am using a local cache (and I have also tried using a local<br>
>>>>>fasta<br>
>>>>>file), I am at a loss to explain why I would get completely different<br>
>>>>>results by these two approaches for a LoF call. As mentioned above, my<br>
>>>>>cache was downloaded using the installer script and default settings<br>
>>>>>(ensembl-tools release-82).<br>
>>>>># Running Offline<br>
>>>>>#Uploaded_variation Consequence IMPACT LoF<br>
>>>>>10_126691951_C/- - 10:126691951 - frameshift_variant HIGH LC<br>
>>>>>NON_CAN_SPLICE_SURR<br>
>>>>>10_126692023_G/- - 10:126692023 - frameshift_variant HIGH LC<br>
>>>>>NON_CAN_SPLICE_SURR<br>
>>>>># Running Online<br>
>>>>>#Uploaded_variation Consequence IMPACT LoF<br>
>>>>>10_126691951_C/- - 10:126691951 - frameshift_variant HIGH HC<br>
>>>>>10_126692023_G/- - 10:126692023 - frameshift_variant HIGH HC<br>
>>>>>I appreciate that neither VCFCols.pm nor LoF.pm were developed by your<br>
>>>>>team, but I would be very grateful if you could help me on these<br>
>>>>>issues<br>
>>>>>as I have been struggling to get VEP customised for my needs for some<br>
>>>>>time<br>
>>>>>now. In regards to issue 1, I believe a lot of your users would<br>
>>>>>benefit<br>
>>>>>from a tool that could grab the original VCF headers in the VEP<br>
>>>>>output, and<br>
>>>>>in regards to the second issue, there must be something strange going<br>
>>>>>on in<br>
>>>>>regards to compatibility with the downloaded caches and the online<br>
>>>>>databases but I am at a loss to explain it.<br>
>>>>>Many thanks in advance<br>
>>>>>Alex Beesley<br>
>>>>>Telethon Kids Institute<br>
>>>>>Perth, Western Australia<br>
<br>
<br>
_______________________________________________<br>
Dev mailing list <a href="mailto:Dev@ensembl.org">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a href="http://lists.ensembl.org/mailman/listinfo/dev" rel="noreferrer" target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a href="http://www.ensembl.info/" rel="noreferrer" target="_blank">http://www.ensembl.info/</a><br>
</blockquote></div><br></div>