Skip to main content
4 of 8
added 341 characters in body; added 133 characters in body; added 102 characters in body
Sky Sanders
  • 12.1k
  • 3
  • 32
  • 60

Paging /users by rep is unstable and returns corrupted results

This is a critical bug. Please read carefully.

Teaser: portions of the results from 2 adjacent pages of the same query. user_ids are in the order recieved.

...

10
7
4
3
2
53

...

and

...

53
7
2
3
4
16

...

Interesting, eh? It gets more interesting if you paid attention when I said that these are subsets of two adjacent pages of the same paged operation.

So not only is the sort indeterminant, it is non-repeatable. Which means the results are unusable.


So, I am paging through users with rep min 99 descending and am getting many duplicate records in subsequent pages.

The total user count has not changed for the site and it is 12am so I have to imagine that there is not enough rep activity to shift 100's of users 100's of places in the 1/5 of a second it takes me to get the next page.

Something is hinky.

The way I qualified this as a bug is with the dupe filter I had emplaced to catch the very, very slim chance that a user loses rep or enough users pass him just as I am getting the next page and he ends up on the next page as well.

This should be a very infrequent occurrence.

In any case, I catch the dupe and go and get the original, compare them, they are the same, I examine the raw json and sure enough, they are in the json. and in no particular order.

Then I notice that they all have the same rep. In the case of the sample urls shown, 101.

The creation dates are unique.

It appears that the sort is on a single field, reputation, and is non-repeatable, making the paging operation unreliable.

Consider - I can catch an arbitrary number of duplicates in subsequent pages, but what about the reciprocal number of items that were replace by the dupes? Are they going to be in the next page or did they get shifted behind me while I was paging? Or is it simply arbitrary? Who knows.

In any case, the sort should not be on a single field that is not unique, like reputation.

To make a sort like this reliable you have to back your reputation sort up with perhaps creation_date and to avoid any problems with mass creations follow up with display_name.

So, here is some pseudo-sql that would result in a reliable sort for the query shown. Remember that reputation desc is the default sort for that route, thus is not sent in the query string.

select * from users 
where rep < int.max and rep > 99 
order by rep desc, creation_date , display_name 

As it is, the results are unusable.

I would hope that this can serve not only as a bug report on this particular route and sorting but also as a heads up that all permutations need to be checked for expected behavior.

samples

Of course these urls will return slightly different data as time goes on, but I have confirmed and repeated these results many times.


http://api.stats.stackexchange.com/1.0/users?max=2147483647&min=99&page=1&pagesize=100

user_id (you can pull the url yourself to see the json)
= = =
8
5
74
159
88
87
36
61
13
90
28
41
69
81
62
66
68
190
174
56
191
55
23
138
6
217
38
183
72
229
54
252
131
237
75
114
71
80
171
198
175
157
173
58
169
57
85
59
209
220
184
181
179
177
176
170
164
163
161
160
156
155
153
152
147
143
141
35
33
30
29
21
19
18
16
12
10
7
4
3
2
53
51
50
43
27
65
64
135
134
102
94
84
129
127
126
123
122
121
116



http://api.stats.stackexchange.com/1.0/users?max=2147483647&min=99&page=2&pagesize=100

user_id (these are the dupes)
= = =

53
7
2
3
4
16
18
19
21
27
10
12
29
30
33
35
84
94
102
67
76
116
121
122
123
126
127
129
141
143
147
152
153
155
156
103
60

Sky Sanders
  • 12.1k
  • 3
  • 32
  • 60