It's two different buffs, and every time you do their respective combo they refresh the timer.
In long fights such as club bosses, you'll basically keep duet up at all times since you're comboing over and over, with ridiculous atk speeds from stacking duet, Elusive, covenant, her sephira and the purple atk speed treasure.
For EC bosses, you'll be able to stack 1 of the first and 2 of the second, or the entire thing for blue bosses assuming you want to go with that strat and not something like yuzu/miku.
For FFT, you'll also fully stack duet before the boss drops for the one shot, unless it's horse then you do other stuff.
It's a fantastic passive to run in any situation where you'll have time to stack it up.