How do I go about creating transactions for the customer/vendor?

You can create a transaction for a contact by following these steps,

• After creating the customer, click on New Transaction under the Customer Details Page to start a new transaction.
• Every type of transaction supported in Zoho Books can be created directly from here.