1. A recent positive trend is that many journals pay referees for finishing a report within a certain number of days. I approve. However, my paper is currently at such a journal, which has limit of 4-6 weeks (range given to protect the anonymity of the journal). After that, a referee gets nothing. What I wonder, then, is if a referee misses the deadline, they no longer have any incentive to write a report sooner rather than later. I also wonder if this doesn't make them more likely to delay. If the deadline was 30 days, and you wake up on day 31, and realize you've missed a payday, it seems you might be less likely to submit the report on day 31, feeling like a chump. A phase out of the payment over several months would likely be more effective. Here's another idea, why I'm throwing them out: referees should be given an option to have a larger payoff with a short duration of a referee report -- say, two weeks -- while also agreeing to pay if they don't finish a report within two months, with a fee increasing incrementally each month. And would need to provide their credit card information in advance. The fee for backing out could be set to be even worse than submitting a report at each date, to make this incentive compatible.
2. Another problem is that much research is funded by the government, or by universities paid for by tuition dollars, and yet academic research is not a public good. Academic journals make bank. First, academics write papers and submit them to journals, and not only do they do this free of charge, but they actually pay for the right to have the fruits of their free labor published, for the benefit of the journal owners. We must do this, of course, as our academic careers depend on publishing in fancy journals. Our careers are also helped by prestigious editing positions at journals, which we are also happy to do for free or low wages. Referees then, also, typically work for free or at least below-market wages. Next, the journals charge lots of money to the same universities which pay their professors for access to the same research the universities pay professors to produce. It's a great business to be in, if you can get it. The issue is that the most prestigious journals have near monopolies -- the tradeoff is that one Top 5 publication is seen as being worth 4-5 publications in Top Fields (at least for your first one) -- a crazy ratio, but the profession is obsessed with rank and status. The solution here is some form of collective action. For example, the government should tax academic journal profits, and could use the proceeds to help fund education, or it could regulate the fees that journals charge for libraries or the public to access the research. If the Ivy League, or the UC system were to spell out that only research published in open access journals or which obey some other criterion would result in tenure, this could be a start. The problems and solutions here are so obvious I'm hesitant to write it down. A dream of mine is also to devise a journal ranking system which penalizes journals for bad behavior (e.g., for high fees to libraries, or high submission fees), so that departments could change the ranking system which they use to award tenure and promotion decisions, in order to better incentivize journals.
3. A third problem is that referees, and even journal editors, have limited incentives to try and do a good job, much less a timely job. One thing I propose here is for journals to ask authors for feedback on how the referee and editorial process went. Clearly, when someone gets a negative referee report, they will give someone a bad ranking, but overall referee scores could be computed controlling for the recommendation of the report. Partly, I think this would be good for purely therapeutic reasons, even if the journals didn't use this for anything. When you're pissed off after getting yet another clueless report, you now have an outlet -- you can kill them in your referee Eval! However, the journals could band together to create a referee ranking. Editors and other referees could also rate referees. There are certainly cases where referees say crazy things, and this is acknowledged by not only the authors, but the editor and other referees. And yet, the said referee pays no penalty at all. On the contrary, they may receive the benefit of not being asked to referee again for some time. If a system were designed to give feedback to departments: "your professor X has a history of providing unprofessional reports", or simply: "here is the rank of your APs in terms of their prowess writing referee reports", this could change incentives for the better.
4. Another problem is that journals have no incentives to publish comment papers, which tend to reflect poorly on the journal, and which also tend to be poorly cited. Thus, in the ranking system I devised above, I would penalize journals that don't accept comment papers on papers published in its own journal.
