3

I notice that even if I send accept-encoding=gzip the response does not seem to be compressed. ( it is late and i was conflating 'accept' with 'accept-encoding') Should we expect, in the 1.0 timeframe to be getting gzipped responses and therefore expect to need to set that header?

I would like to know in order to make some design decisions.

ADDITIONAL INFO

Kevin, here is some data for consideration in response to your response.

From XMLHttpRequest

GET /0.8/badges?key=NLIXkFCt8Eu-oq1Mzw3pgg&jsonp=jsonp1276425901091 HTTP/1.1
Host: api.stackoverflow.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.70 Safari/533.4
Referer: http://soapi.info/code/js/stable/soapi-explore-beta.htm
Cache-Control: max-age=0
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: __qca=P0-473080984-1269917382233; usr=t=puUW6SFMp0Sx&s=IgtyKFGnWUKz; __utmz=140029553.1276424207.947.185.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=sztupy%20codeproject; __utma=140029553.1848660693.1269917382.1276411100.1276424207.947; __utmc=140029553


HTTP/1.1 200 OK
Server: nginx
Date: Sun, 13 Jun 2010 11:17:08 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Cache-Control: private
Content-Encoding: gzip
X-AspNetMvc-Version: 2.0
X-RateLimit-Max: 10000
X-RateLimit-Current: 9223
X-AspNet-Version: 2.0.50727
Content-Length: 5918

[binary data]

From Silverlight ClientHttpRequest which does not actually allow setting the accept-encoding header and does not support gzip.

GET /0.8/users/242897/comments/160173?key=qgAq_KfDu0KYzlNG-qaTuw HTTP/1.1
Accept: */*
Accept-Encoding: identity
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E)
Host: api.stackoverflow.com
Connection: Keep-Alive


HTTP/1.1 200 OK
Server: nginx
Date: Sun, 13 Jun 2010 11:17:03 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Cache-Control: private
X-AspNetMvc-Version: 2.0
X-RateLimit-Max: 10000
X-RateLimit-Current: 9232
X-AspNet-Version: 2.0.50727
Content-Length: 2430

{
  "total": 3,
  "page": 1,
  "pagesize": 30,
  "comments": [

I was mistaken in the details of my initial report as I was setting the 'accept' header, not accept-encoding (this is not supported by silverlight).

But the end result is that I am not sending (not able to send) 'accept-encoding=gzip' and I am getting uncompressed responses from the server. This, while ultimately not what we want, is the correct result.

The issue I am having is that using the browser stack in silverlight will enable gzip but is crippled in every other area and is sub-optimal to say the least.

Using the .net stack enables, amongst other critical functionality, parsing of the rate limit headers. This is a known issue in SL3 and is not to change. I am not sure about SL4, I haven't dug deep enough yet.

And regarding 'you can't get a response that isn't compressed; you can only choose between gzip & deflate' - the sessions above disagree, and rightly so.

If the server was to disregard accept-encoding headers and force compression it would simply 'break the internet'.

So I guess we can chalk this question up as a head's up that SL3 clients are likely to be pulling text.

I am going to spend (likely waste) some time working with the browser stack and try to get an acceptable implementation going but I don't have high hopes.

MO'N'BETTA INFO

Groggy me, making a lot of noise but saying little: The reason that I have to use the ClientHttp stack is that the api has no crossdomain policy file and the BrowserHttp stack fails.

So, I guess the question can evolve to "Is there a reason that a cross domain policy file is not present?"

Silverlight thinks that flash's crossdomain.xml format is yummy.

crossdomain.xml

<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
  <allow-http-request-headers-from domain="*" headers="*"/>
</cross-domain-policy>

ClientHttpWebRequest does NOT request nor accept compressed response. Note the 'Transforms' UI.

XHR

alt text

ClientHttpWebRequest

alt text

2

1 Answer 1

1

Used fiddler to double-check* this.

Everything I get back from the API is definitely gzip compressed.

For that matter, you can't get a response that isn't compressed; you can only choose between gzip & deflate. Default is gzip.

The API always sets the Content-Encoding response header, so you might be the victim of your library decompressing the response for you.

*Triple-checked: I ran our test cases without setting Accept-Encoding but leaving the gzip decompression code in.


Bare minimum request using the world's worst stackoverflow clone (w/ gzip off):

Header:

GET http://api.stackoverflow.com/0.8/users/242897/comments/160173?sort=votes&order=desc&page=1&pagesize=30&key=iv1qWOxcwkaKg53RJIXh-A HTTP/1.1
User-Agent: WorldsWorstStackOverflowClone
Host: api.stackoverflow.com
Connection: Keep-Alive

Response:

HTTP/1.1 200 OK
Server: nginx
Date: Sun, 13 Jun 2010 13:44:21 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Cache-Control: private
Content-Encoding: gzip
X-AspNetMvc-Version: 2.0
X-RateLimit-Max: 10000
X-RateLimit-Current: 9473
X-AspNet-Version: 2.0.50727
Content-Length: 625

[gzip'd byte stream]

This is a quite old question, but just to be 100% clear.

The API always responds with gzip'd content.

There are cross policy domain files on the API, StackAuth, and sstatic for Flash and Silverlight.

9
  • it is in the late hours for me and my initial report was specious but there may be something in the update to consider. Jun 13, 2010 at 11:46
  • @code poet - I'm like 99% sure that Siverlight is just hiding its handling of the Content-Encoding header from you, we jump through some hoops to never return uncompressed content. Send me the code and I'll investigate to be sure. Jun 13, 2010 at 13:29
  • Kevin, the headers are from fiddler. Jun 13, 2010 at 18:44
  • 1
    And PLEASE, do not succeed in your efforts to ignore internet standards by forcing gzip. That would simply be all kinds of wrong on all kinds of levels and not what I would expect from the Stack Overflow team. Jun 13, 2010 at 18:52
  • @code poet - yeah... you're gonna have to accept gzip compressed content. We're not compromising on that, its 2010, the future is now, etc. etc. Technically, we should be 406'ing non-compressed requests; but that breaks tons of proxies (we found this out the hard way) so we do something non-standard to work around others who are breaking standards. There's no perfect solution. Also - I think I might know what's causing this gzip problem, will try and have a fix out for it tomorrow evening. No promises on the time-frame, and I would still like your code to test with. Jun 14, 2010 at 4:19
  • I am not sure you are picking up what I am putting down: Silverlight ClientHttp stack does not support gzip. That is the code we are talking about here. But Soapi is here: [email protected]/bitpusher/soapi - I am pushing on documentation and last minute adjustments but I generally only push compilable code. Open the vs9 solution - I haven't reviewed the vs10 solution in the last few hours and since I am linking files I promise it is broken. Soapi.SL.Tests is what you are looking for. Jun 14, 2010 at 4:29
  • If you force gzip you will break Silverlight 3 (and possibly 4) clients. Unless there is a 'strict in what you issue, relaxed in what you accept' feature in the SL stack that I haven't tested. When I get a moment I will build a server that only pushes gzip and see how the SL stack handles it. Jun 14, 2010 at 4:32
  • @code poet - behavior is "honor accept-encoding, but if omitted assume gzip". Either way a Content-Encoding is set accordingly. Jun 14, 2010 at 4:47
  • 1
    So, silverlight is sending accept-encoding = identity which equates to no encoding and apparently the servers are honoring it. Which is a good thing. If it stops, a lot of (valuable) hours of development time for an unknown number of devoted SO aficionado will be lost. Jun 14, 2010 at 5:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .