Once you reach Ashina Castle for the first time, head to the right of the Ashina Castle Idol and then up the path to find a group of Ashina Samurai. After defeating them, head up and around to the left to find Fujioka hiding out. Agree to defeat the Nightjar on the nearby rooftops and then return to him to deliver the news.
Return to the Dilapidated Temple to find Fujioka has set up shop where he will remain for the rest of the game. If you missed the first step then Fujioka will arrive here of his own accord after defeating the boss at the top of Ashina Castle.
After defeating Hanbei with the Mortal Blade, speak to Fujioka again to learn some more information. Additionally, after the [Sculptor] leaves the Temple during the final stages of the game, speak to him to get his thoughts on the matter.