I owe the group an apology:<div><br></div><div>Here is my foot: *me eating it*</div><div><br></div><div>I am afraid that my frustrations with wget were due to a problem with my linux 'group' permissions not playing nice and resulting in this ftp error:</div>
<div><br></div><div><div>wget -m <a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/">ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/</a> </div><div>--2011-10-31 15:19:08-- <a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/">ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/</a></div>
<div> => `<a href="http://ftp.ensemblgenomes.org/pub/metazoa/release-9/.listing">ftp.ensemblgenomes.org/pub/metazoa/release-9/.listing</a>'</div><div>Resolving ftp.ensemblgenomes.org... 193.62.197.94</div>
<div>Connecting to <a href="http://ftp.ensemblgenomes.org">ftp.ensemblgenomes.org</a>|193.62.197.94|:21... connected.</div><div>Logging in as anonymous ... Logged in!</div><div>==> SYST ... done. ==> PWD ... done.</div>
<div>==> TYPE I ... done. ==> CWD (1) /pub/metazoa/release-9 ... done.</div><div>==> PASV ... done. ==> LIST ... done.</div><div><a href="http://ftp.ensemblgenomes.org/pub/metazoa/release-9">ftp.ensemblgenomes.org/pub/metazoa/release-9</a>: No such file or <a href="http://directoryftp.ensemblgenomes.org/pub/metazoa/release-9/.listing">directoryftp.ensemblgenomes.org/pub/metazoa/release-9/.listing</a>: No such file or directory</div>
<div><br></div><div>I had never encountered this error before and it was not obvious what was wrong. All the stackoverflow/googling I did blamed that response on the robots.txt file which is why I thought wget was being blocked.</div>
<div><br></div><div>Fixing my "group" rules has fixed the problem. Thank you for your help.</div><div><br></div><div>Even though I am quite ashamed about such an oversight leading to such a diatribe on my part with the wget problem, I would like to humbly maintain my suggestion about trying to get ALL the ensembls on the same page at least with file version naming conventions to allow unified file handling from species to species and sub-site to sub-site.</div>
<div><br></div><div>Enjoying a beautifully bitter slice of humble-pie:</div><div><br></div><div>Gus</div><div><br><br><div class="gmail_quote">On Mon, Oct 31, 2011 at 1:54 PM, W. Augustine Dunn III <span dir="ltr"><<a href="mailto:wadunn83@gmail.com">wadunn83@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">In short:<br>
why is this not enabled?!<br>
<br>
But with a few more words and hopefully a bit less fragrant frustration bleeding though (I'll TRY anyway):<div><br>
I have spent a very frustrating few hours trying to mirror a few species from one of the metazoa.ensembl releases (Aedes,Culex,Anopheles,Dmel). You may prefer that people go to the html ftp interface (<a href="http://metazoa.ensembl.org/info/data/ftp/index.html)" target="_blank">http://metazoa.ensembl.org/info/data/ftp/index.html)</a> and repetitively sit at their desk and right-click>download for every file needed for every species needed but this very annoying and results in an un-organized set of files that then need to be put BACK in the nice little order that they were ALREADY in before I had the misfortune of realizing that I needed to try to get data from you guys. <br>
<br>
I appreciate that for the lay-person it is useful to have point-and-click a-la-carte downloads but why prevent those that can actually make the MOST use of your data (people who know their way around a cmd line and understand the need to keep data in a predetermined/organized structure) from using standard methods to access your data in a way that is able to be automated and scripted for reliable updating in the future?<br>
<br>
Actually, on that note can you PLEASE standardize the names of the files included in diff "release" folders? Exp: in relese9 the gtf for anopheles is <a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/gtf/anopheles_gambiae/anopheles_gambiae.AgamP3.62.gtf.gz" target="_blank">ftp://ftp.ensemblgenomes.org/pub/metazoa/release-9/gtf/anopheles_gambiae/anopheles_gambiae.AgamP3.62.gtf.gz</a> which is GREAT because it gives me some idea of which DB version the gtf is based from. BUT in release 10 its now named: <a href="ftp://ftp.ensemblgenomes.org/pub/metazoa/release-10/gtf/anopheles_gambiae/Anopheles_gambiae.AgamP3.gtf.gz." target="_blank">ftp://ftp.ensemblgenomes.org/pub/metazoa/release-10/gtf/anopheles_gambiae/Anopheles_gambiae.AgamP3.gtf.gz. </a> This gives people no idea which gene-build this gtf represents bc AgamP3 refers to a genome assembly version NOT a gene-build. <br>
<br>
Basically, PLEASE PLEASE PLEASE decide on a standard way to name files so that they reflect what version of data they contain. PREFERABLY just use Ensembl's conventions! If you guys are gonna split these orphan geneomes out into their own little "kids table at Christmas dinner" the LEAST you could do is try to make that as un-obstructive to the science that relies on this data as possible. Its not like people work only in metazoa.ensembl OR the "blessed" normal ensembl. Why put easily avoidable stumbling blocks in the way? All that we should have to do is use a different url. All other internals should be made to behave identically. Anything else is directly contributing to important science NOT GETTING DONE. Not because the data is unavailable or tainted but because the scientists are spending all their time dealing with cryptically broken scripts and trying to learn another schema if there even IS a cohearent new one to begin with.<br>
<br>
PLEASE make this better! A the VERY least please let us use standard tools like wget/curl/rsync to pull whole directory structures down intact. PLEASE PLEASE PLEASE.<br>
<br>I am sorry if I have offended. it is NOT my intention. I am simply incredibly frustrated because I have to work in both these domains and simple things like easy access and standardized naming conventions does NOT seem like that hard of a thing to get right.</div>
<div><br>
Gus Dunn <br clear="all"><div><br></div> -- <br><font color="#888888">
<br>
W. Augustine Dunn, III<br>
Ph.D. Candidate<br>
Laboratory of Dr. Anthony James<br>
Department of Molecular Biology and Biochemistry<br>
University of California, Irvine<br>
<a href="tel:%28949%29%20824-3210" value="+19498243210" target="_blank">(949) 824-3210</a> - Lab<br>
<a href="tel:%28949%29%20824-8551" value="+19498248551" target="_blank">(949) 824-8551</a> - Fax
</font></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><font face="Arial, Helvetica, sans-serif" size="3"><font size="2">In science, "fact" can only mean "confirmed
to such a degree that it would be perverse to withhold provisional
assent." I suppose that apples might start to rise
tomorrow, but the possibility does not merit equal time
in physics classrooms. <br><i>-Stephen Jay Gould</i></font><br>
</font><br>
</div></div>