[Bioperl-l] SearchIO speed up

[Sendu Bala]

Sean Davis wrote:
> 
> 
> On 8/14/06 10:00 AM, "aaron.j.mackey at gsk.com" <aaron.j.mackey at gsk.com>
> wrote:
> 
>> I'm failing to understand, sorry.
>>
>> The UNIX utility "more" (or "less" if you prefer) is a pull parser; it
>> reads the stream as much as it needs to satisfy the current iteration (the
>> next iteration occurring when the user asks for an additional screen or
>> line).  It does not copy data from a pipe into temp storage.
>>
>> That said, you can't use "more" to page backwards in piped content (unless
>> your "more" is keeping a buffer, which some do).
>>
>> So, I agree that you will need some form of storage for the *current*
>> information to be parsed (and must process all of the stream necessary to
>> obtain all such information), but not for any of the information yet to be
>> accessed.
> 
> I hesitate to try to "clarify", but this is as much for my own good as for
> that of others.  I think the distinction here is between "random access",
> which is probably not necessary for Blast parsing

Not 'necessary', but if you don't make use of random access you lose a 
lot of the possible advantages of having a pull-parser (ie. will be 
forced to parse and store more data than the user strictly needs).

It'll become clearer when I make some code available, can continue 
discussion then.