This often works better than adding everyone as friends, because you know which group you're jumping into and don't have a bunch of sparse single player sessions in your list. However, I've seen it work well when Group Leaders work as hubs for others to meet up. It's often easier to just add everyone as friends and jump into their open session from the Online Session menu.
Personally I think this system could use a bit of work. You should jump right into their session. The final option is to Join the Invited Session. Finally, go to a quest counter/board/handler.
When you're back in game you'll see a message saying you've received an invite to their session.
So feel free to join as many groups as suits you. Important! You can join games from any Steam Group you've joined, not just your Primary Group. Take a look at the options, you'll see "View Steam Group Information".