Only the Swamp Island has the Swamp Biome. There is no Swamp Village island.
You can get Willow Sapling, Mushroom and Cattail on the Swamp Island. Sheep, Cow and Swamp Slime can also be found here. Zombie, Zombie Archer and Swamp Zombie spawn at night.
There is no entrance to the Caves on the Swamp Island. You need to craft Ladder Down to access to the Swamp Caves.