Are there penalties for these supplies?

If the supplier was aware or was supposed to be aware that the buyer was not registered for VAT at the time of supply, the supplier and the buyer will both be liable for any tax due and penalties applied on the supply.