While it’s easy to focus on more apparent things, like arm strength and mobility, there’s a less obvious factor that figures into a quarterback’s success: his relationship with his head coach. Those two need to be on the same page; together, they decide the fate of most games. During their time together in New England, few duos had a better working relationship than Tom Brady and Bill Belichick.

Although the two combined to life the New England Patriots to incredible heights, the two weren’t exactly singing Kumbaya behind the scenes. According to Randy Moss, Bill Belichick once “clowned” Tom Brady during a team meeting by comparing him to a high school quarterback.

Tom Brady and Bill Belichick formed an unstoppable team

These days, it’s tough to imagine the New England Patriots as anything other than the class of the AFC East. That status is due, in large part, to Bill Belichick and Tom Brady.

When both men arrived in New England, though, neither seemed like a world-beater. Belichick had made a name for himself as a talented defensive coordinator, but didn’t do much in his previous head coaching job; Brady was a skinny sixth-round pick who seemed destined for a career as either a backup quarterback or a journeyman. They, of course, would turn into one of the top duos in modern NFL history.

During their time in Foxborough, Belichick and Brady proved to be the perfect match. Both men cared about winning above all else; from there, they developed both a comfort level and a connection over the years.

“Tom Brady is an extension of Bill Belichick, and Belichick an extension of Brady,” Brian Billick told ESPN’s Greg Garber. “The offense and defense is totally integrated—not only structurally, but philosophically, too. I don’t know if we’ve had a relationship like this ever in the league.”

Bill Belichick clowned his star playing during a film session

During their time together, Bill Belichick and Tom Brady formed quite a bond. That connection, however, didn’t prevent the coach from occasionally calling out his quarterback.

According to Randy Moss, he and Brady failed to connect on a short pass during a two-minute drill. When it came time for film study, that moment wasn’t forgotten; Bill Belichick was ready to make an example of his quarterback.

The head coach played the two-minute drill, calling special attention to the failed pass that ended the drive. “He left the offense with, ‘I got a Hall of Fame wide receiver. I got a Hall of Fame quarterback. They can’t not complete a five-yard out,’ in front of the whole team,” Moss explained on ESPN’s “DangerTalk” podcast. “And he said, ‘If I want a quarterback to complete five-yard out, I’d go down there and get the local high school quarterback at so-and-so high school.”

Moss explained that he remembered thinking, “Wow, man, he just clowned Tom Brady in front of everybody.” The receiver then looked over toward the quarterback, wondering how he would respond.

“So I’m looking at Tom’s face to see if he took that as like ‘OK Bill, I’m going to show you,’ or did he chuckle,” Moss continued. “Man, Tom Brady is sitting up there with his legs crossed, arms folded with a look on his face like, ‘[F–k you].”

That dynamic made the Patriots great but might have driven Tom Brady away

While it might seem harsh in retrospect, it’s tough to argue with Bill Belichick’s methods; he and Tom Brady pushed each other to be their best, and the pair won six Super Bowls together. His tough love, however, could have also pushed the quarterback to try something new.

Although there hasn’t been any official confirmation as to why Tom Brady left New England, most of the theories center around Bill Belichick in some capacity. Some believe the coach’s hard-line negotiations inspired the quarterback to seek a big-money deal elsewhere; others, like Joe Montana, believe that Brady was frustrated by his lack of input into organizational decisions, which seems to point the finger at Belichick.

It is worth noting, though, that not every theory focuses on Belichick and Brady butting heads. The quarterback’s father, for example, said that things in New England became “a touch more boring” after 20 years.

Even if Tom Brady did get sick of working Bill Belichick, though, one thing is clear: their tough-love relationship helped create one of the NFL’s top dynasties.