Are you obligated to join up with the group in front or back if you are a single?

I played with 2 other guys last week (we were 3 singles), the 4 some in front of us lost 3 guys after the front 9, which made this guy a single player. The course was packed that day, 4 somes all the way through, so there was no way he was playing through.

Every hole on the back 9 was 2 to 3 groups deep. But this single refused to join up with us. For the first couple of holes, we waited patiently for him to ask us to join him after he would tee off, but he wouldn't even acknowledge us, we then asked if he wanted to join us at 11, he politely said no thanks. Now, I know there is nothing that says anybody has to join up with anybody else, but isn't it just common courtesy?

And after feeling shunned by this guy he decides, at 17, to join up with the 4 some in FRONT of him, making them a group of 5 and played out the last two holes with them. To each his own...I guess.