3

I cant seem to download the JSON for the questions on SO, I cant get the HTML and JSON from other addresses, but the SO API always returns jibberish.

Here is my code:

public void BeginGetAllRecentQuestions()
    {
        WebClient client = new WebClient();
        client.Headers = new WebHeaderCollection();
        client.DownloadStringCompleted += DownloadRecentQuestionsStringCompleted;
        client.DownloadStringAsync(new Uri("http://api.stackoverflow.com/1.0/questions?key=MYKEYHERE", UriKind.Absolute));
    }

private void DownloadRecentQuestionsStringCompleted(object sender, DownloadStringCompletedEventArgs e)
    {
        var resultsString = e.Result;
        Console.WriteLine(resultsString);
    }

I just want basic JSON text returned, but all I ever get is, what looks like, strangley encoded text, which is not right.

I have tried other JSON URL's and they work fine. Its just this URL that fails.

Can someone please try out this code and help me out?

1
  • 2
    Re. other JSON URL's [...] work fine: this is an artifact of a deliberate decision of those in charge of the API design to violate the respective HTTP specification, see e.g. this answer for some background - please note that the questioner disagrees with this decision, and so do I: while real world issues could justify substantial deviations from standards, the API team has never answered the question why other APIs in the wild seem to work just fine without doing so ... Sep 16, 2010 at 10:07

2 Answers 2

5

Update

Ok, I see from comments that you are using phone7 - the story is slightly different than mainstream silverlight as there is ONLY ClientHttp stack and you will need to manually decompress, as you have found out.

I use Silverlight.SharpZipLib for this purpose.

Good luck.

Here is a method that will work in Silverlight and Phone 7. You can use it for API calls as well as others as it will check for gzip and respond accordingly.

private static string GetResponseText(WebResponse response)
{
    string responseText;
    using (Stream stream = response.GetResponseStream())
    {
        if (stream == null)
        {
            throw new Exception("response stream null");
        }

        byte[] data;

        var buffer = new byte[32768];

        using (var ms = new MemoryStream())
        {
            int read;
            while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
            {
                ms.Write(buffer, 0, read);

            }
            data = ms.ToArray();
        }

        // check for magic numbers
        if (data.Length > 2 && (data[0] == 31 && data[1] == 139))
        {
            using (var ms = new MemoryStream(data))
            using (var gzip = new GZipInputStream(ms))
            using (var reader = new StreamReader(gzip))
            {
                responseText = reader.ReadToEnd();
            }
        }
        else
        {
            responseText = Encoding.UTF8.GetString(data, 0, data.Length);
        }

    }
    return responseText;
}

Previous answer

ALL output from the API is gzipped.

There are two stacks available in silverlight:

  • BrowserHttp - supports transparent gzip - see Soapi.CS source code for a working example, but I will try to provide an isolated spike below.

  • ClientHttp - does not support gzip - you will need determine if the content is gzipped and decompress manually - using perhaps Silverlight.SharpZipLib. But really there are other issues that surface when trying to use the ClientHttp stack so I do not recommend this approach.

So, if you have a line of code in your app resembling

WebRequest.RegisterPrefix("http"..., ClientHttp....

remove it and you should get better results.

4
  • @mark - check update for code example. Sep 16, 2010 at 6:47
  • but you know, WebRequest.RegisterPrefix did not help me, I had to use Silverlight.SharpZipLib library for decoding gzip content. still many many thanks to you. :) Nov 5, 2010 at 22:12
  • @zain - yes, that suggestion was from the initial answer - if you are using phone7 there is only ClientHttp so manual decompression is required. don't thank me, thank the guy that created silverlight.sharpziplib with a review on codeplex. oh, wait... that is me. ;-) seriously, a review of slziplib on codeplex would be greatly appreciated. Nov 5, 2010 at 22:37
  • oh really it is You.....! wow great. I sometimes wonder that how people create one thing for usage when it is not available in the core. like Silverlight does not provide anything for gzip decompression but you did. (Y) :) Nov 5, 2010 at 22:53
1

Random thing to try, but I wonder if this is a compression issue? i.e. WebClient (on silverlight) is failing to decode GZIP of DEFLATE encoded data. What http headers do you send/receive? (via a network trace)

3
  • hmm, not sure, but the WebClient class doesnt let you change headers on a GET request, perhaps im using the wrong class?
    – Mark
    Sep 16, 2010 at 5:55
  • ok, I got it, so the response was compressed, thats what the issue was... I am doing this on Windows Phone 7 and did not expect that...
    – Mark
    Sep 16, 2010 at 6:07
  • 2
    marc - the default silverlight http stack (BrowserHttp) is opaque in that it is simply using the browser and in doing so, compression is handled transparently. If you make the conscious decision to use the embedded clr stack (ClientHttp) by registering an IWebRequestCreator, you lose support for compression and must handle it manually. In the OP case, winphone7, the silverlight port supports only ClientHttp. So, the onus is on the user to 1)determine if the data is compressed and 2) decompress it. just an FYI Sep 16, 2010 at 11:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .