[MOBY-dev] getting RDF resources in the compressed format

[Edward Kawas]

No, we can still use java libraries (I am doing this with the java RDF
parsers):

public static InputStream getBufferedInputStream(URL url) throws
MobyException {
	try {
	    HttpURLConnection urlConnection = null;	
	    urlConnection = (HttpURLConnection) url.openConnection();
	    urlConnection.setRequestProperty("User-Agent",
"jmoby-extended/1.0");
	    urlConnection.setRequestProperty("Accept-Encoding", "gzip,
deflate");
	    urlConnection.setDefaultUseCaches(false);
	    urlConnection.setUseCaches(false);
	    
	    BufferedInputStream is = null;
	    if
(("gzip").equalsIgnoreCase(urlConnection.getContentEncoding())) {
		// handle gzip encoded content
		is = new BufferedInputStream(new
GZIPInputStream(urlConnection.getInputStream()));
	    } else {
		is = new
BufferedInputStream(urlConnection.getInputStream());
	    }
	    
	    return is;
	} catch (IOException e) {
		throw new MobyException("Error creating input
stream:",e.getCause());
	}
}

Eddie

-----Original Message-----
From: moby-dev-bounces at lists.open-bio.org
[mailto:moby-dev-bounces at lists.open-bio.org] On Behalf Of Paul Gordon
Sent: February-27-08 10:29 AM
To: Core developer announcements
Subject: Re: [MOBY-dev] getting RDF resources in the compressed format

The question is whether URL.openStream() sends such a header.  Probably 
not, in which case CentralXXXImpl will need to use Apache httpclient or 
something similar instead...

Edward Kawas wrote:
> Okay,
>
> I set up mod_deflate on my local linux installation and tried it out.
>
> Some results:
> Service instance RDF:
> 	Original size: 7052579bytes
> 	Compressed size: 305092bytes
> 	Savings: ~96%
> Object RDF:
> 	Original size: 2864668 bytes
> 	Compressed size: 219478 bytes
> 	Savings: ~92%
>
> I know that it is obvious that we will get big savings by compressing text
> files ... The question was how to implement this ...
>
> Here is what I did:
> 	1. enabled mod_deflate for apache
> 	This differs from installation to installation, but for me, running
> ubuntu, all I had to do was create 2 sym links in
/etc/apache2/mods-enabled/
> for the files deflate.conf and deflate.load in
/etc/apache2/mods-available.
> 	2. edit the conf file
> 	Again this differs, but what I did was edit deflate.conf and put the
> following inside:
>
> <IfModule mod_deflate.c>
> 	  # mime types to compress
>         AddOutputFilterByType DEFLATE text/html text/plain text/xml
> 	  # support for older browsers
>         BrowserMatch ^Mozilla/4 gzip-only-text/html
>         BrowserMatch ^Mozilla/4\.0[678] no-gzip
>         BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
>         BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
> </IfModule>
>
>
> 	3. I restarted apache
>
> More doc available at
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html
>
> Now when a request comes in with the header 'Accept-Encoding: gzip' apache
> compresses the RDF.
>
> Eddie
>
> -----Original Message-----
> From: moby-dev-bounces at lists.open-bio.org
> [mailto:moby-dev-bounces at lists.open-bio.org] On Behalf Of Martin Senger
> Sent: February-26-08 4:35 PM
> To: Core developer announcements
> Subject: Re: [MOBY-dev] getting RDF resources in the compressed format
>
> Many thanks, Paul.
>
> I believe the solution we agreed upon was that if the client sends an
>   
>> "Accept-encoding" HTTP header with x-gzip
>>     
>
>
> Yes, that was what I vaguely remembered but was not sure about details.
> [BTW, is there a way to search the whole biomoby email archive?]
>
> How does client know what was sent from the server? Is is in the HTTP
> Content-type header? If so, what mime-type should be there?
>
> Eddie, it would be nice to have this feature implemented on the registry
> site (so other registry providers can copy it, as well). Would you
consider
> it please?
>
> Cheers,
> Martin
>
>
>   
_______________________________________________
MOBY-dev mailing list
MOBY-dev at lists.open-bio.org
http://lists.open-bio.org/mailman/listinfo/moby-dev