[Bioperl-l] BioPerl CVS EOL Inconsistency

Nathan Haigh nsh104 at york.ac.uk
Mon Nov 1 10:32:10 EST 2004 

To clarify what i think the problem is, i've taken snippets from the 
following page:
See the explaination at 
http://www.tortoisecvs.org/faq.html#brokenlineendings

Essentially, the BioPerl CVS server should store files with the UNIX 
style end-of-line (EOL) char (LF). When files are checked out/commited 
with a windows based CVS client (CVSnt), EOL conversions takes place (LF 
<-> CR LF) automatically. However, if you try to commit dos files with a 
non-windows CVS client, this conversion doesn't occur so the CVS  server 
now has files that contain CR LF EOL chars.

This causes the following problems:

    * If you check out the file on UNIX (where no conversion is
      performed), the line endings will be <CR><LF> and not <LF>, which
      is wrong.
    * If you check out the file on Windows using CVSNT, it will convert
      each <LF> to a <CR><LF> to set the corret line endings for
      Windows. Unfortunately, the server copy of the file already *had*
      a <CR> before the <LF>. As a result, the local line endings will
      be <CR><CR><LF> - the file is totally screwed up!

General rules for avoiding this:

    * It's highly recommended not to access your sandbox (that was
      checked out with TortoiseCVS, CVSNT or WinCVS) using other CVS
      clients, especially not with UNIX-style CVS clients (like Linux's
      or Cygwin's) because of line ending incompatibilities.
    * Furthermore, you have to be careful when you've checked out a
      module using UNIX line endings. You must not edit those files with
      a Windows text editor that overwrites the UNIX line endings with
      DOS line endings, or else you'll eventually get that additional
      <CR> at the end of each line.

Therefore, files in BioPerl CVS need to be run through dos2unix and 
recommited - i could supply a list of files that are affected (up to 
around 50 files) if someone wanted to commit them!

Nathan


Nathan Haigh wrote:

>Ok, here's what I think I've found! If anyone knows more about this that I do, please feel free to interject!
>
>I have found that the files stored in BioPerl CVS have different end-of-line chars, and when CVSnt does conversions on these chars,
>it results in odd end-of-line chars, that I assume were affecting my previous "nmake test" using the files downloaded by CVSnt.
>
>As an example, if you look at Bio::SeqIO::Interpro, this file contains the EOL char "0D 0A" (Hex) which is the DOS EOL char (CR LF).
>When CVSnt downloads this file; it convert it to "0D 0D 0A" (Hex) which is not a standard EOL char (it is actually CR CR LF).
>Whereas Bio::SeqIO::GenBank contains the EOL char "0A" (Hex) which is the standard UNIX EOL char (LF). CVSnt converts this correctly
>into "0D 0A" (Hex) which is the DOS EOL char (CR LF).
>
>I'm not sure how these different EOL chars occur, because as far as I understand CVS, a CVS client should convert the EOL chars to
>those used by their OS when the files are downloaded, and then converts them back to those used on the CVS server during a commit. I
>think the only way this mixing of EOL chars can happen is if someone uses a different CVS client or OS to commit the files.
>
>On second thoughts, it is probably due to the fact that people will submit patched files from several OS's to BioPerl for CVS
>commits, but these files do not get converted from DOS/Mac to Unix before commits.
>
>Is there anything that can be done to bring all the EOL chars back into line as I'd really like to be able to use CVS on windows!?
>
>Thanks
>Nathan
>
>
>  
>
>>-----Original Message-----
>>From: bioperl-l-bounces at portal.open-bio.org [mailto:bioperl-l-bounces at portal.open-bio.org] On Behalf Of Nathan Haigh
>>Sent: 01 November 2004 08:37
>>To: bioperl-l at bioperl.org
>>Subject: RE: [Bioperl-l] nmake test
>>
>>Hmm, this seems to be a problem associated with CVSnt which I used to obtain the latest BioPerl from CVS - I got fed up with
>>    
>>
>having
>  
>
>>to download the entire BioPerl package in tarball for small updates.
>>
>>The latest tarball tested without errors, I'm now trying CVSnt again to see if I can reproduce the problem! On the first attempt I
>>noticed that several of the BioPerl files contained '?' at the end of every line, so I suspected some mix up in the end of line
>>characters; but replacing these files with the relevant ones from the website did not rectify the problem (unless I missed some).
>>
>>I'll post again if I can replicate the problem, in case it is any help to someone else. But as far as I can see the tarball seems
>>    
>>
>to
>  
>
>>test without errors!
>>
>>Nathan
>>
>>
>>    
>>
>>>-----Original Message-----
>>>From: bioperl-l-bounces at portal.open-bio.org [mailto:bioperl-l-bounces at portal.open-bio.org] On Behalf Of Nathan Haigh
>>>Sent: 31 October 2004 11:36
>>>To: bioperl-l at bioperl.org
>>>Subject: [Bioperl-l] nmake test
>>>
>>>Hi, just to let you know the latest BioPerl from CVS is now giving the following errors on WinXP, and I am trying to debug them:
>>>
>>>Failed Test   Stat Wstat Total Fail  Failed  List of Failed
>>>-----------------------------------------------------------
>>>t\Index.t      255 65280    41    0   0.00%  ??
>>>t\Pictogram.t  255 65280     3    3 100.00%  1-3
>>>t\ProtPsm.t    255 65280     5    5 100.00%  1-5
>>>t\flat.t       255 65280    16    3  18.75%  14-16
>>>t\psm.t        255 65280    48   48 100.00%  1-48
>>>2 subtests skipped.
>>>
>>>Nathan
>>>
>>>
>>>_______________________________________________
>>>Bioperl-l mailing list
>>>Bioperl-l at portal.open-bio.org
>>>http://portal.open-bio.org/mailman/listinfo/bioperl-l
>>>      
>>>
>>_______________________________________________
>>Bioperl-l mailing list
>>Bioperl-l at portal.open-bio.org
>>http://portal.open-bio.org/mailman/listinfo/bioperl-l
>>    
>>
>---
>avast! Antivirus: Outbound message clean.
>Virus Database (VPS): 0444-3, 29/10/2004
>Tested on: 01/11/2004 10:21:43
>avast! is copyright (c) 2000-2003 ALWIL Software.
>http://www.avast.com
>
>
>
>---
>avast! Antivirus: Outbound message clean.
>Virus Database (VPS): 0444-3, 29/10/2004
>Tested on: 01/11/2004 10:23:10
>avast! is copyright (c) 2000-2003 ALWIL Software.
>http://www.avast.com
>
>
>
>
>
>_______________________________________________
>Bioperl-l mailing list
>Bioperl-l at portal.open-bio.org
>http://portal.open-bio.org/mailman/listinfo/bioperl-l
>  
>

More information about the Bioperl-l mailing list