If you work on a public holiday, penalty rates apply under the relevant award. For casual employees under the Horticulture Award, work on a public holiday is paid at 225% of the minimum hourly rate (the casual loading is already included in that rate). That is not the same as a generic double time or double time and a half promise.
Casual employees who do not work on the public holiday are generally not paid for that day. Permanent employees who do not work may still be entitled to the public holiday under the National Employment Standards, depending on their roster.
Check Fair Work’s Pay and Conditions Tool for your classification and the relevant public holiday.