Firstly, keep in mind that this details stuff different from the relatively superficial Enemy Bestiary. Demon fusion is an ability unlocked early in the game with the purchase of the Cathedral of Shadows app.
Per its description, demon fusion requires the melding together of two or more demons to create a new one altogether. This is not necessarily a more powerful demon; often, it's simply an average of the component demons' powers.
Demon fusion works not unlike quick evolution. The parents will effectively die off while their new child has traits of them. This includes statistics to a point, as well as the ability to put through certain skills that you handpick at the time of fusion.
However, if the resultant demon's level is too high, the fusion cannot normally be done; this level is typically equal to your own.
How do you know what kind of demon pops out? Race is a primary factor in determining race - two races will yield a certain other race.
The individual monster species is determined by the component demons' species as well. If you don't fuse with the Element race, both factors tend to be unique. If you do, you have the same race, but a stronger or weaker species, really.
There are a few special kinds of fusions as well. Special Fusions, as they are aptly named, can use more than the usual two demons to create super-powerful demons, often ones that are bosses in the game.
They cost a lot of Macca to make as well. To be able to unlock them, you must usually first beat the corresponding boss, if one exists, as well as discover (and sometimes Scout in) the component demons.
The other kind of special fusion is an Error Fusion - it is a complete accident that results in a demon behaving unaccordingly to the rules of demon fusion. Let me give an example. Early in the game, I mixed together two demons from the upper layers of Naraku, and it was an error.
This resulted in a Level 30 Basilisk - far beyond my own level, of a race that shouldn't have been possible with the races I used. Some demons are supposedly only obtained this way. Keep in mind that it won't always work in your favor, and it's a random occurrence that only Special Fusions are immune to.