From the Pokemon Center, head east to the end of the road. Now turn south and head down that road. Just before the blue-roofed building to your west is some obscured stairs. Move down them to the water below and head west until your character reaches water.
When this happens, use Surf to move out onto the water to your west. Now head west under the bridge. Once you're past it, move north under the next bridge. Once you're past that, you'll reach some stairs. Move onto the land and up the stairs. Grab the Rare Candy there!