The goats play a central, yet little understood, role in the avodah of Yom Kippur (no longer performed, but part of the Tefilos).  The chicken plays a role in the Ashkenazic practice of Kapparos immediately before Yom Kippur– also a little understood ritual.
Read more on Yeshiva World News