Why?
I got tired of searching for the same duplicate questions over and over again.
What?
These are a couple of scripts I threw together in order to cut down search time for common questions and vote to close targets. dupes.txt is the output in a flat text file of the form:
Link: url Count: number of times this was cited as the original for a dupe closure
Link: url Count: number of times this was cited as the original for a dupe closure
...
It's sorted in descending order of duplicates
Usage: ./dupes.sh posts.xml
Yields: ./dupes.txt
dupes.sh
#/bin/bash
cat $1 | grep ' ClosedDate\=\"' | grep 'Possible Duplicate' | awk -F 'Possible Duplicate:</strong><br>
 <a href="' '{print $2}' | awk -F '">' '{print $1}' | grep questions > dupes.txt
python dupes.py
dupes.py
lines = []
with open('dupes.txt', 'r') as dupeList:
for line in dupeList.readlines():
lines.append(line.rstrip())
links = {}
for line in lines:
try:
links[line] += 1
except:
links[line] = 1
thelinks = []
for link in links.keys():
thelinks.append((link, links[link]))
thelinksort = sorted(thelinks, key=lambda alink: alink[1])
thelinksort.reverse()
with open('dupes.txt', 'w') as dupeList:
for line in thelinksort:
dupeList.write('Link: ' + line[0] + ' Count: ' + str(line[1]) + '\n')
