Does your paper or the draw of the referee matter more?
On the x-axis is a measure of what referees agree on about a paper (which may be true quality of the paper but could also be writing style or sexiness). On the y-axis is what they would report to the journal editor. Depending on the journal, only a small fraction of papers are not rejected. Clearly, papers with high agreeable rank (1.0) are more likely to receive an R&R, but there is a great deal of noise in the process. (PS: with multiple referees, usually one negative view is enough to eliminate the paper.)
Ergo: Both the draw of the referee and the paper quality matter greatly. (Not shown, some referees are intrinsically more generous than others.)
The evidence is from a random-assignment prestigious conference, but the paper shows that the same estimates apply to (non-random assignment) submissions to top econ and finance journals, too.