The short answer is yes, but it does depend on how much we are talking about.
You will have to take him to court obviously but first you must start a paper trail by issuing him with an "invoice" stating quite clearly what you believe he owes you and giving him a set time to respond failing which you will enter a claim against him. Even if you are successful in getting a judgement, there is no guarantee you will ever receive the money unless you take certain steps to secure your debt.
It's a large minefield and obviously you may have received goods and services which he could clearly claim were beneficial to you, so it would be difficult for you to prove that you should have ALL your money back.
If he is a business I would be tempted to inform him (in writing sent by Recorded (signed for) Delivery) stating the extent of your dissatisfaction) that the work he has done is unacceptable and unless he reimburses you with 'X' money in recompense (being mainly the cost of work by others to put HIS work right) you will have no option but to report him to Trading Standards.
The threat is always better than the action, but you must be able to prove that his work was unsatisfactory and have expert testimony to prove it.
As for the request for 'cash' to 'avoid VAT' be very careful what you allege if you can't prove it...or you may end up on the wrong end of a lawsuit. Keep that stuff up your sleeve for when you need it (but again, only if you can prove it).
'English Law' is quite straight forward but does not rely on any emotional or 'just' concept inasmuch as you will have to prove you were wronged.
Step 1. Prove you have a contract (in writing) which alludes to the quality, scope and cost of works.
Step 2. 1/3rd party expert statement on quality of work done and cost of putting it right if unacceptable standard..
Step 3. Letter by recorded delivery stating what you believe you are owed in compensation for bad/non works and giving say 14 days to respond, or 'else'
Step 4. Action depends on response to '3; above.
Step 5. Burn his house down if not satisfied. No, just kidding.
You may decide it's just not worth it after all and put it down to experience. But please, first give the guy a chance (in writing) to put it all right for you and then you will appear to be the 'reasonable' party in any consequential legal action.
Hope this helps.
PS Always attempt to acquire the moral high ground in any legal dispute.