Take the case of North Sea cod: once the 'poster child' for overfishing and all that was wrong with European policy, it is now recovering strongly and likely to be certified as sustainable next year.
Fish is as tasty and popular as ever, but no one seems to like the policies that regulate the industry behind it. For decades, European management of fisheries has been lambasted by fishers, conservationists and scientists, including us.
The centrepiece of this system, the EU's Common Fisheries Policy, is particularly unpopular. Some scientists even argue it is designed to fail. Opponents blame it for not only mismanaging Europe's highly productive seas, but also for giving away 'our fish', with the subject recently taking centre stage in an unlikely viral Brexit campaign video.
You might think that the chance to take back control of the fish in UK seas would be one of the most solid reasons to vote 'Out' in June's referendum on EU membership. So what's the catch?
First, the idea that fish in British waters have been fished into near-extinction by pesky foreign boats simply doesn't match up with reality. At least not anymore.
Yes, fish numbers aren't what they were in the time of Moby Dick. However, a recent analysis of 118 years of statistics revealed the vast majority of the decline occurred prior to the Common Fisheries Policy's implementation in 1983. In fact, the policy is now overall helping, not harming, the country's fisheries.
Since EU policy was reformed in 2002, the health of many fish stocks has improved. By 2011 the majority of assessed fisheries were considered to be sustainably fished. Take the case of North Sea cod: once the 'poster child' for overfishing and all that was wrong with European policy, it is now recovering strongly and likely to be certified as sustainable next year.
Who actually owns 'our' fish?
Ownership of UK fishing quotas is controversial and often misunderstood. After total EU fishing limits are decided by the Council of fisheries ministers, it is up to each member state to distribute its share among its own fleet.
This is not an EU decision. The fact that a single giant Dutch-owned vessel nets a quarter of the English quota (6% of the UK total) might be shocking, especially considering the UK's quota is in theory shared between more than 6,000 vessels, but the UK government could easily change how it allocates fish. In fact, the alternative allocation systems suggested by some pro-Brexit groups are already in place elsewhere in Europe.
Mackerel, herring, cod and other commercial species are all highly mobile, and move easily across borders, especially in places such as the North, Celtic and Irish Seas, where 'exclusive economic zones' are jammed together like sardines in a can.
So unlike more isolated countries such as Iceland and Norway, the UK was always going to have to share its fish with its neighbours, especially as we moved into an era of global maritime regulation.
Fencing out foreign fishermen
A post-Brexit UK might still have to agree quotas with its neighbours, but could it prevent foreign boats from fishing in its waters? Maybe. But only with huge investment in monitoring and control public bodies such as the Marine Management Organisation (MMO) - organisations which are being cut at present.
Whether the UK would want this sort of escalation is a different question, as it would also mean British boats could no longer fish in the waters of other European nations. This is a major concern in the fishing industry as 20% of the fish caught by the UK fleet is landed elsewhere in the EU.
The reality is that a Brexit would require a complete re-negotiation of fishing rights, with uncertain outcomes. Some of these rights extend back to the Middle Ages and banning foreign vessels from UK waters may well be incompatible with international law.
Such negotiations may harm trading relationships with Europe. At present the UK exports around 80% of its wild-caught seafood, with four of the top five destinations being European countries.
Remaining in the EU also has big benefits for the marine ecosystems that the fishing industry ultimately relies on. The Habitats Directive protects key habitats and species such as reefs and Atlantic salmon, while the Water Framework Directive and Marine Strategy Framework Directive commit EU members to restore and protect the environment.
It seems unlikely that the UK's current Conservative government, at least, would continue similarly progressive measures after a Brexit.
It's no surprise the 'In' campaign is gaining support from a range of environmental groups - the weight of evidence is on their side. In contrast, many fishermen have strong feelings about the EU, but the main industry organisations and decision makers are remaining neutral.
We've come a long way since the bad old days of excessive quotas and widespread illegal fishing. As things become more sustainable, fish numbers are rebounding, leading to increasing UK fishing quotas and growing profits - now the highest in the EU!
The history of the EU's fishing policy is one of criticism and improvement. It is therefore unclear why the UK would want to abandon ship at this point.
Bryce Stewart is Lecturer in Marine Ecosystem Management, University of York.
Griffin Carpenter is Visiting Lecturer in Environmental Economics, Oxford Brookes University.