<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000066">
<div class="moz-cite-prefix">Hello,<br>
<br>
I just added the chr5.vcf file that is causing me the crashes to
the repo.<br>
Forget about the weird vcf format. First columns are standard the
rest are not.<br>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<a
href="https://github.com/guillermomarco/vep_plugins_71/blob/master/chr5.vcf">https://github.com/guillermomarco/vep_plugins_71/blob/master/chr5.vcf</a><br>
<br>
On 05/14/2013 08:37 AM, Guillermo Marco Puche wrote:<br>
</div>
<blockquote cite="mid:5191DBB1.1060509@sistemasgenomicos.com"
type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Hello,<br>
<br>
I'm not really sure which one of those plugins is causing the
fork error. I cannot recreate it now running each one of them
separately.<br>
<br>
Here are both:<br>
<br>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
<a moz-do-not-send="true"
href="https://github.com/guillermomarco/vep_plugins_71">https://github.com/guillermomarco/vep_plugins_71</a><br>
<br>
They also slow the calculating consequences process a lot.
aa_grantham_distance.pm is just a hardcoded plugin from one of
the biologists in my work. It was just a pure copy paste and
adaptation to make it work as a VEP plugin. Maybe the problem is
in the matrix definition every time the sub routine is called.
I'm not running out of memory nor CPU. I'm currently using it
with 2 threads and buffersize of 500 for a 5000 variant vcf
file.<br>
<br>
I'm my honest opinion, I think one (or even both) of those
plugins are slowing so much the calculating process that
sometimes the fork just dies. Like when you have a timeout
during to heavy network traffic. So when you use them together
with lot of other plugins like Condel, Consequence, etc.. they
may be causing the process to handle and die.<br>
<br>
Best regards,<br>
Guillermo.<br>
<br>
On 05/13/2013 03:55 PM, Duarte Molha wrote:<br>
</div>
<blockquote
cite="mid:CAGqkoE+MjhwtB4nVk9kF6Ah0SMU9jg-VDC9nMLfsm63OvOedLg@mail.gmail.com"
type="cite">
<div dir="ltr">I also get this error... it is so prevalent and
so difficult to pinpoint what is causing it that I have given
up on forking my annotation process.
<div><br>
</div>
<div>I do think it is related to the number of forks. It seems
to crash less often if you use a low number of forks...
anything above 5 will undoubtedly crash the script at least
in my experience.</div>
<div><br>
</div>
<div style="">Cheers</div>
<div style=""><br>
Duarte</div>
</div>
<div class="gmail_extra"><br clear="all">
<div><font style="background-color:rgb(255,255,255)"
color="#999999">=========================<br>
Duarte Miguel Paulo Molha <br>
</font>
<div><font style="background-color:rgb(255,255,255)"
color="#999999"> <a moz-do-not-send="true"
href="http://about.me/duarte" target="_blank">http://about.me/duarte</a>
<br>
=========================</font></div>
</div>
<br>
<br>
<div class="gmail_quote">On Mon, May 13, 2013 at 2:50 PM, Will
McLaren <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:wm2@ebi.ac.uk" target="_blank">wm2@ebi.ac.uk</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Guillermo,
<div><br>
</div>
<div>Test each plugin individually until you find the
one that causes the error. It is highly unlikely that
a particular combination of plugins is causing the
crash.</div>
<div><br>
</div>
<div>Check that there are no "print" (to STDOUT or
STDERR) statements in your plugin - forking assumes
that code remains silent otherwise it will throw
errors like this.</div>
<div><br>
</div>
<div> Also, check what, if anything, is cached between
runs of your plugin. If you are caching things (for
example to avoid re-querying a database), you may need
to write storable hooks to ensure the data is getting
cached between forks - see <a moz-do-not-send="true"
href="https://github.com/ensembl-variation/VEP_plugins/blob/master/ProteinSeqs.pm"
target="_blank">https://github.com/ensembl-variation/VEP_plugins/blob/master/ProteinSeqs.pm</a>
for an example.</div>
<div><br>
</div>
<div>If you still have no luck, send me the code and an
input file that recreates the problem.</div>
<div><br>
</div>
<div>Regards</div>
<div><br>
</div>
<div>Will</div>
</div>
<div class="gmail_extra"> <br>
<br>
<div class="gmail_quote">
<div>
<div class="h5">On 13 May 2013 13:18, Guillermo
Marco Puche <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:guillermo.marco@sistemasgenomicos.com"
target="_blank">guillermo.marco@sistemasgenomicos.com</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div class="h5">
<div bgcolor="#FFFFFF" text="#000066"> Hello,<br>
<br>
I've started to recently having problems with
VEP script while using different plugins (most
of them own plugins).<br>
<br>
<pre>2013-05-13 13:59:44 - Connected to core version 71 database and variation version 71 database
2013-05-13 13:59:44 - Loaded plugin: vcf_input
2013-05-13 13:59:44 - Loaded plugin: biobase
2013-05-13 13:59:44 - Loaded plugin: aa_grantham_distance
2013-05-13 13:59:44 - Loaded plugin: flanking_sequence
2013-05-13 13:59:44 - Loaded plugin: Condel
2013-05-13 13:59:44 - Output fields redefined (37 defined)
2013-05-13 13:59:44 - Starting...
2013-05-13 13:59:45 - Read 3888 variants into buffer
2013-05-13 13:59:54 - Reading transcript data from cache and/or database
[===============================================] [ 100% ]
2013-05-13 14:02:38 - Retrieved 6463 transcripts (0 mem, 0 cached, 13743 DB, 7280 duplicates)
2013-05-13 14:02:38 - Calculating consequences
[===================================> ] [ 78% ]
ERROR: Forked process failed
</pre>
I'm not getting any other error message. So I
cannot debug properly. I thought my plugins
were OK but it's seems they don't. I think the
problem occurs when I use
"aa_grantham_distance plugin" together with
"flanking_sequence". I've no idea what could
be causing this.<br>
<br>
I'm running VEP on verbose mode but I can't
get any usefull information. How could I debug
that?<br>
<br>
Best regards,<br>
Guillermo.<br>
<br>
</div>
<br>
</div>
</div>
_______________________________________________<br>
Dev mailing list <a moz-do-not-send="true"
href="mailto:Dev@ensembl.org" target="_blank">Dev@ensembl.org</a><br>
Posting guidelines and subscribe/unsubscribe info: <a
moz-do-not-send="true"
href="http://lists.ensembl.org/mailman/listinfo/dev"
target="_blank">http://lists.ensembl.org/mailman/listinfo/dev</a><br>
Ensembl Blog: <a moz-do-not-send="true"
href="http://www.ensembl.info/" target="_blank">http://www.ensembl.info/</a><br>
<br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</blockquote>
</body>
</html>